PHP App Engine

http://groups.drupal.org/node/36750

Very similar to setting up an instance with a single EBS drive, you can also fairly easily do a RAID configuration. This leads to better performance usually and higher reliability.

http://blog.reddit.com/2010/01/why-did-we-take-reddit-down-for-71.html

Part of our setup uses what we call a “permacache”, which uses Memcachedb. Memcachedb is Memcached with a built-in permanent storage system using BDB. One of the “features” of this system is that it saves up its disk writes and then bursts them to the disk. Unfortunately, the single EBS volumes they were on could not handle these bursting writes. Memcachedb also has another feature that blocks all reads while it writes to the disk. These two things together would cause the site to go down for about 30 seconds every hour or so lately.

http://news.ycombinator.com/item?id=1037993

http://aws-musings.com/how-to-create-an-ebs-image-from-an-existing-ec2-instance/

Amazon recently announced a new feature which allows you to boot from an ebs volume. But it doesn’t provide any tools to convert your existing AMIs to this new type of image. There is no easy way to create an ebs image from scratch. There are some posts that explain how to convert your existing AMI into this new type of image using ec2-unbundle and dd (a linux utility). I am going to take a little different route and explain how we can create an ebs image from an existing instance. It’s fairly simple to create a new image using dd from an existing instance.

http://stu.mp/2009/12/disk-io-and-throughput-benchmarks-on-amazons-ec2.html

We’ve been ironing out kinks in our production environment for the last few weeks and one of the things that worried me was if these assertions were true. So, I set out to run a fairly comprehensive test of Disk IO and throughput. I ran hdparm -t, bonnie++, and iozone against ephemeral drives in various configurations along with EBS volumes in various configurations.

For all of my tests I tested the regular ephemeral drives as they were installed (”Normal”), LVM in a JBOD setup (”LVM”), the two ephemeral drives in a software RAID0 setup (”RAID0″), a single 100GB EBS volume (”EBS”), and two 100GB EBS volumes in a software RAID0 setup (”EBS RAID0″). All of the tests were ran on large instances (m1.large). All tests were ran using the XFS file system.

http://news.ycombinator.com/item?id=986445 – YC thread

http://blogs.adobe.com/livecycle/2009/12/running_livecycle_on_amazon_ec.html

One of the major annoyances with Amazon’s Elastic Compute Cloud (EC2) has been its behavior when you shut down one of the instances. All changes you made were immediately and irretrievably lost. You had to be a programmer to backup your changes to Amazon’s Simple Storage Service (S3). If you decided to keep your instance running overnight so that you could continue working the next day, you incurred hefty charges for the 8-12 hours of night time during which you do not even use it.

Amazon then introduced the Elastic Block Store (EBS) service which essentially is your own hard disk where you could save things that would survive a machine shutdown. However, up until a week ago, you could not boot from your EBS volume. Now you can.

http://blog.shutupandcode.net/?p=374

Solr, Zabbix, Selenium

http://debianzone.org/raid-and-lvm-on-amazon-ec2-part-i/

This is the first part of three articles I’m posting for a great storage solution using RAID, LVM and Amazon Elastic Block Store (EBS).

First, you need to choose your RAID implementation. Personally, I prefer to use RAID 5 on Amazon EC2, combined with LVM2.

http://docs.google.com/View?docID=dhh4z6n4_96w387mqhn&revision=_latest

Using LVM + DRBD + NFS + Heartbeat + VTun To Gain Data Persistence, Redundancy, Automatic Fail-Over, and Read/Write Disk Access Across Multiple EC2 Nodes

http://vehera.jsn-server7.com/LiddleBlog/?p=518

Amazon Elastic Block Storage (Amazon EBS) is a new type of storage designed specifically for Amazon EC2 instances. Amazon EBS allows you to create volumes that can be mounted as devices by EC2 instances. Amazon EBS volumes behave as if they were raw unformatted external hard drives and can be formatted using a file system such as ext3 (Linux) or NTFS (Windows) and mounted on an EC2 instance; files are accessed through the file system . They have user supplied device names and provide a block device interface.

http://biodivertido.blogspot.com/2009/11/amazon-ebs-raid-0-postgis-build.html

EC2’s dirty secret

Javier’s post was a great tutorial on building out a PostGIS database on Amazon EC2. We all know EC2, but it does have it’s drawbacks and they are mainly related to disk IO. When using EC2 & EBS with large datasets you can easily run into IO bottlenecks. Individually these are not such a big deal, but when you are conducting global analyses poor disk IO on EC2 & EBS can quickly become a problem.

http://news.ycombinator.com/item?id=943046

We run all of reddit on ec2, which includes a bunch of postgres servers. Each one is running with a single EBS. However, I’ve heard horror stories of people trying to run much less busy databases and having lots of problems, usually with MySQL.

Edit: I forgot to mention that on all the database disks, we use ext2 and noatime. Both decrease the total number of writes necessary, and have very little downside (the biggest being that you have to fsck on a crash).

http://tech.backtype.com/increasing-io-performance-on-ec2ebs

When you start working with large datasets that don’t fit in memory, improving IO performance on EC2 becomes a priority. To do that at BackType, we use software RAID with EBS volumes; with EBS you don’t need added redundancy so you can settle with RAID0:

http://jframeworks.com/blog/2009/11/08/running-mysql-on-amazon-ec2-with-elastic-block-store-another-take/

I have been following ‘Eric Hammod’ article on how to setup MySQL so that, it survives EC2 instance termination or restarts. That article is one of best article but it’s too much for my requirements. So I came up with some more simple steps to host MySQL files on EBS volume. Many parts of Eric’s article still applies like how to create EBS volume and how to create snapshot of EBS.

http://craiccomputing.blogspot.com/2009/08/creating-and-attaching-aws-ebs-volumes.html

AWS Elastic Block Store (EBS) Volumes are arbitrary blocks of storage that can be mounted on EC2 nodes and used like regular filesystems. Here are the basic steps needed to set them up (and the associated gotchas to avoid).

http://www.mooreds.com/wordpress/archives/000549

The biggest attraction of the cloud was the ability to spin up and spin down extra servers as the expected traffic on the site increased or decreased. We choose Amazon’s EC2 for hosting. They seem a bit like the IBM of the cloud–no one ever got fired, etc. They have a rich set of offerings and great documentation.

Exactly.

Fork your AMI
Use Capistrano
Use EBS
EC2 Persistence
Use ElasticFox
Consider CloudFront
Use Internal Network Addressing where possible
Consider reserved instances

Some right, some not.

http://www.ardentsoft.com/blog/tag/lamp-on-ec2

Part 1: Setting up Amazon Web Services
Part 2: Setting up a Persistent Volume
Part 3: Starting and Configuring the Instance
Part 4: Mounting a Persistent Volume
Part 5: Configuring MySQL
Part 6: Configuring Apache
Part 7: Configuring PHP
Part 8: Setting up an Elastic IP Address
Part 9: Setting up Snapshots
Part 10: Generating a Custom AMI

http://serverfault.com/questions/77535/what-is-the-fastest-filesystem-to-use-under-linux-on-ec2-for-mysql

EXT3 will work just fine, but you may want to use EBS volumes instead of the space that comes with your EC2 instance, as that may be faster.

Edit: You can create a software RAID on EBS to boost performance (I’d recommend RAID-10) by using mdadm.

http://www.floydprice.com/2009/10/automatic-elastic-block-snapshots-with-a-cron-job/

Amazon EC2 really is amazing, and the Elastic Block storage is pretty darn good too, however I wish you could automate the snapshot process form the EC2 console.

Before you start make sure you have a JRE:

http://pigeonflight.blogspot.com/2009/08/growing-xfs-partition-on-amazon-ebs.html

Amazon EBS is Amazon’s Persistent Storage service. I think of it as the ability to have a “thumbdrive in the clouds”. I can then “attach” that drive to an Amazon EC2 instance (a machine in the clouds).

Today I needed to add space to a filesystem. The filesystem was 10G XFS partition and I wanted to make it 30G. (don’t try this if you’re not using XFS, some of these notes are XFS specific).

http://vasyas.blogspot.com/2008/10/amazon-ec2-amazon-ebs-java-mysql.html

Устанавливаем на Amazon EC2 и Amazon EBS Java-приложение с MySQL

http://vastpark.posterous.com/moving-to-amazon-web-services

I have been administrating LAMP web servers in some form or another for more than a decade either in support of a business or for personal use. I seem to never have so few projects that shared hosting made sense. So instead, solutions like virtual private servers, managed dedicated hosting and, in some cases, my own rack of equipment have served up my projects over the years.

Most recently I had been using dedicated managed servers on Rackspace and GoDaddy to host a few dozen sites. Because I no longer run my own DNS or mail servers, it made my migration focused on making MySQL and Apache work on AWS. (I no longer host DNS or mail because as I have found services like EveryDNS and Gmail to be far superior.)

http://blog.labnotes.org/2009/10/13/mysql-backups-with-ebs-snapshots/

Amazon EBS snapshots are awesome. You can take snapshots to backup your database, quickly move data between staging and production, run analytics on the latest data, resize volumes and much more.

For Apartly, I’m using MySQL, storing data and log files on a mounted EBS volume. EBS offers persistent, reliable storage (with replication for fault tolerance). Still, nothing can prevent data from disappearing, hence backups.

http://www.john-nousis.com/development/tutorial-running-a-phpmysql-server-on-ec2-with-ebs/

During the presentation I showed how you can start an AMI of fedora 8 base, install apache, php and mySQL and make them all store their data on an EBS volume. This attracted a lot of attention and participants asked me to post this as an online tutorial. This tutorial requires basic knowledge of AWS like running an EC2 instance, creating an EBS volume and assigning an elastic IP to the instance. During the presentation I used ElasticFox which is a firefox plugin that allows you to perform several operations on Amazon through a graphical user interface.

http://ianso.blogspot.com/2009/10/ec2-now-having-actually-played-with-it.html

Consistent IO on EBS is apparently not an option. This is something I think Amazon should fix toute suite, because things like RackSpace (maybe) and NewServers (h.t. etbe) seem to be to stomping all over the EBS I/O figures. In a different context, James Hamilton says “it makes no sense to allow a lower cost component impose constraints on the optimization of a higher cost component”, and assuming that the servers are the expensive part, this is what (IMHO) may make using RDBMSs on EC2 a bit of a PIA long-term.

http://innergysoftware.com/?p=17

Recently I’ve been using a lot Amazon’s Cloud Computing services in the form of EC2 and S3. I’ve set up several scripts that make life easier by automating some of the tasks that need to be done regularly when administering EC2 virtual instances. In the hope that they will be useful to other people, I’ll be posting here some of them. Here is the first in the series.

http://www.freemindsystems.com/amazon-ec2-introduction-useful-info-and-tricks/

I’m sure many people think the cloud is the future, and more or less have migration plannings. Also, some of these people point to Amazon as supplier for their price, stability and capabilities. This first post of EC2 are things you need to know from the standpoint of a sysadmin.

For a System Administrator, Amazon EC2 is a big challenge, mostly because their ephemeral nature:

http://developer.amazonwebservices.com/connect/thread.jspa?threadID=25071

I’ll assume you’ve already set everything up:
1) created the volume (ec2-create-volume)
2) you’ve attached it to your instance
3) Logged into your instance and ran mkfs and mounted the volume

/sbin/chkconfig –add mountec2vol