PHP App Engine

http://www.mysqlperformanceblog.com/2009/10/15/mysql-memcached-or-nosql-tokyo-tyrant-part-1/

A classic example is a simple online game. An online game may only require that an application retrieve a single record from the database. The record may contain all the vital stats for the game, be updated and stuffed back into the database. You would be surprised how many people use this type of system as I run into this type of application frequently. Keeping it simple, ensures that application is generally mean and lean and performs well. The issue is even this simple design can start to have issues as the data size increases and you blow through your available memory. Is there a better architecture? Is there a way to get more scalability out of your database? Is the database even the best place for this data?

percona guys (one of the highest mysql authorities) put some nails in mysql coffin (innocently?)

http://bret.appspot.com/entry/how-friendfeed-uses-mysql

We use MySQL for storing all of the data in FriendFeed. Our database has grown a lot as our user base has grown. We now store over 250 million entries and a bunch of other data, from comments and “likes” to friend lists.

As our database has grown, we have tried to iteratively deal with the scaling issues that come with rapid growth. We did the typical things, like using read slaves and memcache to increase read throughput and sharding our database to improve write throughput. However, as we grew, scaling our existing features to accomodate more traffic turned out to be much less of an issue than adding new features.

http://www.initsix.co.uk/content/how-make-xfs-4-disk-raid-0-partition-ec2-mysql-maximum-disk-io-performance

Use this technique at your own risk. You need to be hot on snapshots, backups and restores to ensure you dont lose everything.

* Raid 0 offers no fault tolerance
* Disk reads and writes can be potentially 400% faster
* Loss of one disk will destroy the entire dataset

http://zuzara.com/blog/2009/11/01/tpcc-mysql-rough-benchmark-for-amazon-rds/

I tried to do tpcc-mysql benchmark for Amazon RDS. Before do that, did the same test for EC2 small instance.

This is a pretty rough benchmark, but I can say EC2 small instance and RDS small instance have the same performances as CPU and memory are the same spec. RDS is about 30% expensive. (EC2=$0.085, RDS=$0.11/h)

http://aws.amazon.com/running_databases/

This page contains the following categories of information.

* Amazon EC2 Relational Database AMIs
* Amazon Relational Database Service (Amazon RDS)
* Amazon SimpleDB
* Storage Practices and Backups
* Security
* Documentation and Case Studies

http://news.ycombinator.com/item?id=943046

We run all of reddit on ec2, which includes a bunch of postgres servers. Each one is running with a single EBS. However, I’ve heard horror stories of people trying to run much less busy databases and having lots of problems, usually with MySQL.

Edit: I forgot to mention that on all the database disks, we use ext2 and noatime. Both decrease the total number of writes necessary, and have very little downside (the biggest being that you have to fsck on a crash).

http://support.rightscale.com/03-Tutorials/02-AWS/02-Website_Edition/2.1_MySQL_Setup

There are two different ways of setting up a redundant MySQL database on Amazon EC2. In both cases, there will be a Master-DB and Slave-DB with replication for failover and recovery. The core difference between each setup is how the backups are saved.

http://jframeworks.com/blog/2009/11/08/running-mysql-on-amazon-ec2-with-elastic-block-store-another-take/

I have been following ‘Eric Hammod’ article on how to setup MySQL so that, it survives EC2 instance termination or restarts. That article is one of best article but it’s too much for my requirements. So I came up with some more simple steps to host MySQL files on EBS volume. Many parts of Eric’s article still applies like how to create EBS volume and how to create snapshot of EBS.

http://www.ardentsoft.com/blog/tag/lamp-on-ec2

Part 1: Setting up Amazon Web Services
Part 2: Setting up a Persistent Volume
Part 3: Starting and Configuring the Instance
Part 4: Mounting a Persistent Volume
Part 5: Configuring MySQL
Part 6: Configuring Apache
Part 7: Configuring PHP
Part 8: Setting up an Elastic IP Address
Part 9: Setting up Snapshots
Part 10: Generating a Custom AMI

http://www.lonelycode.com/2009/10/27/amazon-rds-and-django/

Amazon has recently released a long-awaited feature addition to it’s Web Services infrastructure: Relational Databases. Having for a long time only had SimpleDB on offer (non-relational and weird to implement), it’s always been something of a struggle to get MySQL running on EC2.

Now RDS is actually completely compatible with all mySQL tools, so interfacing it with django is stupid simple – but, for those that want a quick walkthrough, I’ve prepared one below, from getting the service running to getting django to use it:

http://fathomdb.com/about/home

FathomDB is the easiest and best way to run a database on EC2

* Database-as-a-service: pay only for what you use, no up-front costs
* Relational: standard databases like MySQL means nothing to rewrite or re-learn
* Managed: we do the setup, backups and monitoring
* Designed for EC2: out-of-the-box best practice for databases in the cloud
* Deep insight: track and improve performance down to the query level

http://www.techcrunch.com/2009/10/26/amazon-launch-relational-database-cloud-service/

Amazon has launched a hosted relational database service, Amazon RDS, as part of the suite of services available at AWS. The new service is a hosted MySQL database instance with the full capabilities and access rights as a normal self-hosted DB.

http://serverfault.com/questions/77535/what-is-the-fastest-filesystem-to-use-under-linux-on-ec2-for-mysql

EXT3 will work just fine, but you may want to use EBS volumes instead of the space that comes with your EC2 instance, as that may be faster.

Edit: You can create a software RAID on EBS to boost performance (I’d recommend RAID-10) by using mdadm.

http://www.scribd.com/doc/15014619/MySQL-Performance-on-EC2

MySQL Performance on EC2

http://vasyas.blogspot.com/2008/10/amazon-ec2-amazon-ebs-java-mysql.html

Устанавливаем на Amazon EC2 и Amazon EBS Java-приложение с MySQL

http://cloudbuzz.wordpress.com/2009/10/19/load-balancing-mysql-part-i/

We are working towards getting a load balanced MySQL server setup on EC2. So the first of the experimentation is on local physical machines/VMs. So this will be roughly divided into four long parts.

The first step towards achieving this to get the replication working.

http://vastpark.posterous.com/moving-to-amazon-web-services

I have been administrating LAMP web servers in some form or another for more than a decade either in support of a business or for personal use. I seem to never have so few projects that shared hosting made sense. So instead, solutions like virtual private servers, managed dedicated hosting and, in some cases, my own rack of equipment have served up my projects over the years.

Most recently I had been using dedicated managed servers on Rackspace and GoDaddy to host a few dozen sites. Because I no longer run my own DNS or mail servers, it made my migration focused on making MySQL and Apache work on AWS. (I no longer host DNS or mail because as I have found services like EveryDNS and Gmail to be far superior.)

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1074

A Redundant Array of Independent Drives (RAID) is a method of grouping multiple drives together to improve performance or fault tolerance. Because m1.large and m1.xlarge instances have multiple ephemeral stores, you can now use RAID with these instance types.

Important: Using RAID does not guarantee against instance failure and is not a replacement for a sound backup policy. If an underlying drive fails, you should immediately migrate to a new instance.

http://blog.labnotes.org/2009/10/13/mysql-backups-with-ebs-snapshots/

Amazon EBS snapshots are awesome. You can take snapshots to backup your database, quickly move data between staging and production, run analytics on the latest data, resize volumes and much more.

For Apartly, I’m using MySQL, storing data and log files on a mounted EBS volume. EBS offers persistent, reliable storage (with replication for fault tolerance). Still, nothing can prevent data from disappearing, hence backups.

http://www.john-nousis.com/development/tutorial-running-a-phpmysql-server-on-ec2-with-ebs/

During the presentation I showed how you can start an AMI of fedora 8 base, install apache, php and mySQL and make them all store their data on an EBS volume. This attracted a lot of attention and participants asked me to post this as an online tutorial. This tutorial requires basic knowledge of AWS like running an EC2 instance, creating an EBS volume and assigning an elastic IP to the instance. During the presentation I used ElasticFox which is a firefox plugin that allows you to perform several operations on Amazon through a graphical user interface.

http://developer.amazonwebservices.com/connect/entry!default.jspa?categoryID=174

This tutorial describes one approach to using Amazon EC2’s Elastic Block Store (EBS) as a persistent storage mechanism for a production MySQL database server, including snapshot backup and restore.

http://kovshenin.com/archives/cloud-tips-automatic-backups-to-s3/

In a previous post about backing up EC2 MySQL to an Amazon S3 bucket we covered dumping MySQL datasets, compressing them and uploading to S3. After a few weeks test-driving the shell script, I came up with a new version that checks, fixes and optimizes all tables before generating the dump. This is pretty important as mysqldump will fail on whatever step would cause an error (data corruption, crashed tables, etc), thus your uploaded to S3 archive would be kind of corrupt. Here’s the script:

http://stackoverflow.com/questions/1543257/mysql-index-creation-is-slow-on-ec2

There are 11 million rows in this table and the data size is less than 2GB (size of the MYD file.) The index file size is around 600M at the end of the process.

Creating index takes less than 20 minutes on my laptop. But, when I ran the process on Amazon’s EC2 (medium instance), the process took more than 12 hours. During entire process, the disk was superbusy with IO wait (as see by top) between 40-100%. The CPU was mostly idle. I don’t think, the disk on EC2 are that slow.

http://www.catalyzed.org/2009/07/cloud-servers-for-your-perl-app-amazons-ec2-vs-mosso.html#more

This brings into play one of the oddities of EC2. There are issues where, apparently in certain situations, applications are unable to properly determine amount of free RAM on your VM. This problem definitely affects MySQL and is likely to affect other databases as well. If you run into this problem, if you use a complex join (or even a simple join with a lot of records) or your database simply requires more ram than is available, you suddenly wind up dealing with a lot of disk IO.

I have had personal experience with the MySQL / IO problem on EC2 and it is a very hard problem to track down if you don’t know the above. I have not yet had any issues on Mosso, though I have only been using them for a short time.

http://alestic.com/2009/09/ec2-consistent-snapshot

The name of the new program is ec2-consistent-snapshot.

Over the last year as I and my co-workers have been using this code in production systems, we identified a number of ways it could be improved. Or, put another way, some serious issues came up when the idealistic world view of the original simplistic script met the complexities which can and do arise in the brutal real world.