User Tools

Site Tools


inbox:build.with.amazon.ec2

HowTo Build with Amazon EC2

Introduction

If you are one of those people who are impatient (want to compile firmware NOW) but only have one computer and use it for other things (video games) during your waking hours, you may find Amazon EC2 to be a convenient platform for compiling firmware. Whatever the reason, you may at least find it worthwhile as an educational exercise to give this a try.

Cost

Amazon offers a free tier of EC2 services1). You can use a single linux 'micro' instance without having to pay anything for the first year. The largest 'High-CPU' instance is $0.68 / hr. The great thing about EC2 is that you can start your instance as a 'micro' size to download the source and configure your build environment, and then bring it back up as a High CPU for the few hours that you need the horse power. Just remember to shut it down when you are finished.

Creating an AWS Account

This step requires a credit card. Go to http://aws.amazon.com and click "Sign Up Now". Choose a secure password!!

Securing your AWS Account

This is the most important step. You want to secure this account as much as possible. If someone steals your ID and uses it to spin up a bunch of CUDA instances, you can be on the hook for hundreds of dollars per day, not to mention legally responsible for any abuse that might occur from the instance. Why would someone steal your ID for an EC2 cluster? Two words: Bitcoin Mining 2). As ridiculous as it sounds, it could happen to you. So be smart and lock up your AWS account!

Adding 2-Factor Authentication

The best way to secure your account is to use a 2-factor auth token. These can be purchased for $12.99 (+ $2.98 shipping) and are well worth the cost. To buy one, sign into your AWS Console and click Account > Security Credentials. Down below 'AWS Multi-Factor Authentication', click "Learn more about AWS Multi-Factor Authentication'. On the right, under Quick Start click "1. Purchase Device." Click "Add to Cart", then click "Begin Checkout". The rest of this process should be familiar if you shop with Amazon.

In a few days the auth token will arrive by mail. Sign back into your AWS Console and add it under the Security Credentials menu.

You will need this token each time you sign into the AWS Console, as this is your gateway for using their services. You don't need the token to sign into your actual AWS virtual machines once they are created, only for the AWS Console which controls the overall security/firewall and the stuff that can cost you money.

A note about Instance Type and Storage

There are two types of instances. Those with 'instance' storage only, and those with EBS storage. Instance storage is stored on Amazon S3 and is not persistent. That is, your changes disappear after you shut down the instance. Since you are installing packages for the build environment, you will need persistent storage. EBS is the answer. As of writing, it costs around $0.10 per GB per month. With the free tier account, you get 12 GB per month for free. So far I'm using around 3 GB of storage and haven't been charged. So don't be afraid to use EBS.

Launching an Instance

I used a Ubuntu3) 10.04 LTS image, specifically AMI id ami-3202f25b4).

  1. To start this instance, sign into your AWS Console and click > Instances on the left, then click "Launch Instance."
  2. Click the "Community AMIs" tab and search for the instance id ami-3202f25b . Click Select.
  3. Under Instance Details, select your Availability Zone. I chose us-east-1a . Click Continue.
  4. I didn't configure any Advanced Instance Options. Click continue.
  5. I didn't add any tags. Click continue.
  6. Create a new key pair. Give it a name to remember it by. I used "aws_2011-05-30". Save it onto a USB flash drive for maximum security, or somewhere on your local disk.
  7. Create a new security group. Give it a name.
  8. Create a new rule: SSH
  9. Enter your IP address as the source. ex: 8.8.8.8/32
  10. Click Add Rule.
  11. Click Continue.
  12. Click Launch.

Your instance will eventually show up as "Running" in the My Instances overview. Click Refresh till it shows up.

Connecting to your Instance via SSH

I have Windows so these instructions are written for Windows. If you are using Linux, I assume you can figure out how to deal with SSH Keys.

Before you can connect we need to add your SSH Key to PuTTY. If you don't allready have PuTTY, download and install it 5).

  1. Run Puttygen from the start menu. Click Load. Open up the file you saved previously (aws_2011-05-30.pem)
  2. Change the key comment to be something meaningful. I put "aws_2011-05-30"
  3. Enter in a strong key passphrase.
  4. Confirm the passphrase.
  5. Click "Save Private Key". I named it "aws_2011-05-30.ppk"
  6. PPK files are the format that PuTTY Agent uses to import your private key into memory.
  1. Run Putty Agent from the start menu.
  2. Right click on the icon in the tray and select Add Key.
  3. Open your new key and unlock it with your passphrase.

You can now connect to your instance.

Run PuTTY from the start menu. Enter the Instance IP address or hostname. To find it, go back to the AWS Console, right-click on your instance and select Connect. The "Public DNS" will be in that popup under Step 4. Use that hostname.

Now, at this point you can *probably* use the EC2 Instance storage for your build. But if you end up building the entire world, you will probably need a lot more than the 8 GB that came with the instance. So lets create another storage volume.

Creating a second storage volume.

  1. Go back to the AWS Console, on the left under "Elastic Block Store" click Volumes. Click Create Volume.
  2. Enter in the size you think you will need. You are only charged for the amount you use, not the amount you allocate. (I used 20 GB)
  3. Put it in the SAME Availability Zone as your instance. You don't need to select a snapshot. Click Create.
  4. Click Refresh until your EBS Volume is 'Available'.
  5. Right click on the new volume and select "Attach Volume".
  6. Select your Instance ID. Enter a device name. (I used /dev/sdc because the High-CPU instance comes with a /dev/sdb attached)

Back on your instance, you should have a /dev/sdc now. We'll need to format it.

sudo fdisk /dev/sdc
n
p
1
default
default
p
w
You should now have a /dev/sdc1 object. Format it:
sudo mkfs.ext3 /dev/sdc1

Your disk is now formatted and ready to mount.

sudo mkdir /mnt/sdc
sudo mount -t ext3 /dev/sdc1 /mnt/sdc
cd /mnt/sdc
df -h
sudo mkdir openwrt
sudo chown ubuntu:ubuntu openwrt
cd openwrt

OK. You are ready to go. At this point, proceed as you would normally to configure the build environment 6), download the source from SVN 7), and build 8) 9)

Micro or High-CPU?

Once you have downloaded all of the build environment and sources, you are ready to build. At this point you can make a decision. You can build with the micro instance, which is free but slow; or, if you are super impatient, and don't mind paying $0.68/hr for a few hours, you can change the instance to a High CPU version.

Changing to a High CPU version for Building.

  1. Goto your AWS Console and STOP your instance (right click on it and select STOP).
  2. Click refresh until it is stopped.
  3. Right click on it and select "Change Instance Type"
  4. Select "c3.xlarge".

This is the "Extra Large High CPU" instance type with 8 virtual cores with 2.5 ECUs each10). This should compile much faster with the "-j 9" option, like so:

nohup make -j 9 &
Remember, the second you 'start' a High-CPU instance you will pay for 1 full hour for any fraction used. So when you click start you will pay. Don't forget to shut it down when it is finished and change it back to t1.micro. :)

Final Cost

These build times are based on a simple AR71XX configuration with wireless drivers, no LuCI. YMMV.

Cost is based on US East (Virginia) pricing as of 2011-05-31.

Trunk source adds options for Compile certain packages parallelized and Number of package submake jobs (2-512). I enabled this and set it to 8, look for it under Global build settings. (Credit to Mazilo for pointing this out11))

Instance Type Spec Cost / Hr Build Time Total
t1.micro 1 ECU, 1 Core, 650M RAM $0.02 / Free ~14h 00m $0.28
m1.small 1 ECU, 1 Core, 1.7G RAM $0.12 ??? ???
m1.large 4 ECUs, 2 Cores, 7.5G RAM $0.34 ??? ???
m1.xlarge 8 ECUs, 4 Cores, 15G RAM $0.68 ??? ???
c1.medium 5 ECUs, 2 Cores, 1.7G RAM $0.17 ??? ???
c3.xlarge 20 ECUs, 8 Cores, 7G RAM $0.68 ~3h 20m (backfire) $2.72
c3.xlarge 20 ECUs, 8 Cores, 7G RAM $0.68 ~0h 40m (trunk) $0.68
inbox/build.with.amazon.ec2.txt · Last modified: 2011/06/07 22:42 (external edit)