This week I had to fight (again) with our hosting provider in Germany over an IDE hard drive that decided to pass away in the dedicated server. Three days of back-and-forth support tickets, the development server down, and me tearing my hair out because I have little patience. The classic physical infrastructure drama that all of us who manage systems suffer in silence.

And right in the middle of this chaos, I managed to carve out some time in the early hours of the morning to play with the beta access of a service I've been reading about since August on a few forums, and which I think is going to be talked about a lot: Amazon Elastic Compute Cloud, or simply EC2.

Goodbye bare metal, hello on-demand virtualization

Look, the concept of virtualizing operating systems isn't new; we've been using VMware GSX Server for a while to separate test environments. But what Amazon (yes, the book and record store) is doing is insane. Instead of tying you down to a monthly contract of €150 or €300 for a physical machine, they rent you virtual "instances" and charge you per hour of actual use.

Specifically, for 10 cents an hour, they provide you with a Xen-based virtual machine equivalent to a 1.7 GHz Xeon processor, 1.7 GB of RAM, and a 160 GB disk. The best part isn't the ridiculously low price, but the fact that you don't have to call some guy in a suit or sign permanence contracts: you control everything through a pure and simple web API.

Getting into the mud: spinning up a server with one command

To test it, I downloaded the Amazon EC2 Command-Line Tools. They are developed in Java, so you need to have a JRE running on your local machine. The most tedious part is configuring the X.509 certificates and RSA keys for authentication, but once you have your bash environment set up, the magic happens right in the console.

Instead of inserting an installation CD or mounting an ISO, you use an AMI (Amazon Machine Image). Amazon offers a couple of official ones to get started. I used a base image with Fedora Core 4. To spin up the machine, you simply run this:

# Launch a small Fedora Core 4 instance
ec2-run-instances ami-2bb65342 -k my-ec2-keys -g default

In less than three minutes, you throw an ec2-describe-instances and see that Amazon has assigned you a dynamic public hostname. You open your favorite SSH client (PuTTY in my case), point to that host with your private key, and suddenly the prompt gives you back control. You are root on a cloud Linux server, fresh out of the oven.

Once inside, I pulled a yum install httpd mysql-server php to set up a basic LAMP environment. I uploaded a database and the files from one of my WordPress installations, and I was serving pages at full speed in record time. It took no time at all.

Lights and shadows: what future awaits this?

Obviously, not everything is perfect. The biggest technical drawback right now is that the instance storage is ephemeral. If your machine crashes, reboots, or you decide to shut it down, you absolutely lose all the data on the disk that wasn't "baked" into the original AMI. That is, if you put your MySQL database in there and Amazon's physical server reboots, you're back to zero.

Right now, the only viable workaround is to hack together a script and send continuous backups to their other storage service, Amazon S3, using a simple script scheduled in cron. For this reason, for serious production relational databases, I still prefer having a RAID 1 of physical SCSI disks on a machine under my absolute control.

However, as an engineer, the future possibilities blow my mind. Imagine being able to spin up twenty cloned web servers in an automated way using a script just on the day you launch a major campaign, put a load balancer in front of them, and shut them down at night to instantly stop paying. That level of agility is completely unthinkable with the hosting infrastructure we know today.

We are still in a very raw beta, but if Amazon manages to solve the persistent storage issue, IP persistence, and the community starts packaging their own system images, this is going to completely change the architecture of our web applications. Traditional dedicated hosting's days are numbered if they don't get their act together.

Have you guys tried the service yet, or are you still fighting with tech support to get them to replace your broken memory module?

PS: Amazon doesn't pay me.