Our Blog

Aug 22 2012

3 Things You Must Know If You Develop on Amazon Web Services (EC2)

Jamie Begin

Amazon Web Services (AWS) is a great way to begin building highly-scalable, durable web apps almost immediately… er, which is a bit of a problem. If you’re developing applications on AWS, here are three important tips to keep in mind before diving in head first.

Never run anything important on a single EC2 instance.

If take one thing away from this blog post, please remember:

Amazon Web Services is fundamentally different than traditional hosting.

EC2 instances are not durable—they can (and sometimes do) suddenly vanish, taking whatever data you had stored on it. Your application has to be built with this possibility in mind and your AWS architecture should be designed with redundant components to compensate. This would include such things as:

  • storing user-uploaded files in Amazon S3 rather than on the instance
  • using an Elastic Load Balancer in front of a minimum of two application servers, each in a different availability zone
  • using one of Amazon’s cloud-based persistent datastores (SimpleDB, RDS, DynamoDB, etc) rather than installing your own database on EC2. (Or, at least setting up a proper replication strategy if you do have a need to roll your own solution.)
  • maintaining copies of important data entirely outside of Amazon’s cloud platform

Properly developing on AWS requires a change in mindset and a solid understanding of the underlying platform (either through training or experience). Treating AWS as a traditional dedicated server or VPS will result in a deployment that is more complex, less durable, and yet more costly than the alternatives. If your data is important, take the time to read up on AWS best practices or retain the services of an experienced AWS consultant.

Elastic Beanstalk can save you a lot of time.

Elastic Beanstalk is a PaaS-like configuration/deployment framework for Amazon Web Services. Similar to Google App Engine or Heroku, as a developer you can simply upload your source bundle to AWS and application deployment automagically happens—there’s no configuring Apache or even a need to SSH into the instances. Elastic Beanstalk acts as an abstraction layer on-top of EC2 to make it super simple to build autoscaling, durable web apps. Additionally, there are no extra costs for Elastic Beanstalk; you’re simply billed for the Amazon Web Services resources it provisions on your behalf.

I recommend Elastic Beanstalk very frequently to my clients when I consult on their AWS projects. For the most part, it allows their developers to stay focused on the code rather than having to fiddle with server settings. It can save a lot of time and, for plain vanilla LAMP, Java, or Flask/Django apps, it works great. However, sometimes the flip side of “user friendly” is “expert hostile.” If you need to color outside of the lines by using a custom Apache configuration (or ditch it all together for Nginx), you’ll find that you start fighting with the platform. Likewise, if you’re a Ruby on Rails or a Pyramid developer, you’ll also have to look elsewhere. For advanced deployments, Cloud Formation can often be used where Elastic Beanstalk isn’t a good fit.

Easily reduce your Amazon Web Services costs by 40-50%

Buying reserved EC2 instances allow an Amazon Web Services customer to pay a one-time, up-front fee in exchange for a 40-50% discount on the hourly cost for an EC2, RDS, ElastiCache, or DyanmoDB instance. Both one-year and three-year terms are available.

For most production deployments running a web app that’s available 24/7, buying reserved instances is a no-brainer for all but the most seriously cash-strapped start-ups. As an example, let’s calculate the yearly instances costs for a simple, fault-tolerant LAMP stack deployment under Elastic Beanstalk.

Standard, On-demand Instances

Role Instance Type Hourly Cost Up-front Cost Yearly Cost (24/7 utilization)
Elastic Load Balancer $0.025 $219.00
App Server A Small Linux EC2 Instance $0.08 $700.80
App Server B Small Linux EC2 Instance $0.08 $700.80
Database Small Multi-AZ RDS Instance $0.21 $1,839.60
Annual Cost $3,460.20

Heavy Utilization Reserved Instances (1-Year Term)

Role Instance Type Hourly Cost Up-front Cost Yearly Cost (24/7 utilization)
Elastic Load Balancer $0.025 $219.00
App Server A Small Linux EC2 Instance $0.016 $195.00 $140.16
App Server B Small Linux EC2 Instance $0.016 $195.00 $140.16
Database Small Multi-AZ RDS Instance $0.106 $138.00 $928.56
Annual Cost $1,955.88

The cost savings equate to $1,504.32 (or nearly 44% off your Amazon Web Services bill!) over the course of 12 months using reserved instances. Reserved EC2 instances are available in light, medium, and heavy utilization types to fit different usage scenarios. “Heavy utilization” types will get you the most bang for the buck for “always on” instances, but light and medium can also significantly reduce costs (while requiring a smaller up-front payment). It’s worth noting that bandwidth and EBS disk I/O is billed separately and are not affected by reserved instance pricing.

6 thoughts on “3 Things You Must Know If You Develop on Amazon Web Services (EC2)”

  1. John Doe says:

    The last point was the most valuable for myself. One question though: when you use reserved instances and load peaks above the max threshold, does AWS automatically scale up to a third reserved instance, or does it launch another on-demand instance? I’m assuming it launches an on-demand instance, but I figured it was worth asking. Maybe a better question is, what would be the best way to automatically launch reserved instances based on some metric of persistent load?

    1. jamie says:

      Reserved instances are functionally the same thing as standard, on-demand instances. There is no way specify whether to launch an on-demand vs. reserved instance. The billing is just automatically adjusted on the backend, depending on what you’ve allocated.

      If you have two reserved instances running as an app server and another instance is spun up, it will be billed as a reserve instance if you have three reserved instances for that AMI type associated with your account. Otherwise the new instance will just be billed at the standard, on-demand rate. This is why it’s handy to figure out how much up/down scaling your app will do when deciding on light, medium, or heavy reserved instance types. Reserved instance availability and purchasing can be done through the AWS API, so you could actually write your own logic to programmatically do this.

  2. Corey Turner says:

    Great article on EC2!

  3. Thanks for a great post! I think it’s worth mentioning in your calculations that AWS has a free tier which lasts one year, and provides a lot for you. It should cut down on the cost substantially, for a new project which is getting off the ground.

  4. jim says:

    Can you substantiate the claim that:

    “EC2 instances are not durable—they can (and sometimes do) suddenly vanish, taking whatever data you had stored on it.”

    That is a very strong and detrimental claim. Is it possible for you to explain *how* this happens? User error in admin screen? Problems with billing? Amazon’s stated policy that they delete things randomly?

    Without further explanation, it seems like AWS is a non-starter.

    1. Jamie Begin says:

      It can happen for any number of reasons: hardware failure, hypervisor failure, datacenter catching fire, a backhoe operator cutting through fiber, etc. It doesn’t really matter why. “Design for failure” is one of the key architectural concepts when building on AWS. In fact, it’s a mantra used by engineers building any mission critical system, whether that be using physical servers or VMs. However, it’s a lot easier (and cheaper) to do this on AWS since you’re not having to buy and maintain your own hardware.

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe to RightBrain

Recent Posts


Recent Comments