Archive for the ‘HowTos’ Category

Apache LogAnalysis using Pig

Thursday, October 1st, 2009

Analyze your Apache logs using Pig and Amazon Elastic MapReduce.

* Total bytes transferred per hour
* A list of the top 50 IP addresses by traffic per hour
* A list of the top 50 external referrers
* The top 50 search terms in referrals from Bing and Google

You can modify the Pig script to generate additional information.

Grid Engine & Amazon EC2

Wednesday, September 30th, 2009

Good pragmatic presentation

PDF per se.

Updating an AMI’s Kernel and Ramdisk

Wednesday, September 30th, 2009

In order to upgrade your AMI to the latest kernel you will have to launch your existing AMI and rebundle it explicitly overriding the default bundling tools behavior – which is to use the instance meta-data service to determine your current kernel and ramdisk combination OR overriding the launch time kernel and ramdisk parameters.

Drupal EC2 AMI

Wednesday, September 30th, 2009

We’ve created a pre-built AMI that makes it easy to get started with Drupal on EC2. Just start it up and you’re ready to go. And it works with any version of Drupal you’d like.

CouchDB on Ubuntu on AWS

Wednesday, September 30th, 2009

Here’s a little HowTo on how to setup CouchDB on an AWS EC2 instance. But outside of AWS (and EC2), this setup works on any other Ubuntu server, and I suppose Debian as well.

Tips for deploying a LAMP stack on Amazon EC2

Wednesday, September 30th, 2009

If you’re interested in using Amazon EC2 and other services to deploy a LAMP (Linux, Apache, MySQL, and PHP) stack, you will probably find this post invaluable. I spent about three full days migrating all my sites over from a physical dedicated server to an EC2 instance, and what follows are several things I learned during the process.

Rightscale: EC2 Site Architecture Diagrams

Tuesday, September 29th, 2009

The following diagrams will show some of the common site architectures in the Cloud. Depending on your computer resource requirements and budget, RightScale provides you with the flexibility to create a custom architecture that provides the necessary performance and failover redundancy necessary to run your site in the Cloud. Several of the most common architectures are described below. Use one of the setups below as a model or easily customize a setup for your own purposes.

ec2-consistent-snapshot: Creating Consistent EBS Snapshots with MySQL and XFS on EC2

Tuesday, September 29th, 2009

The name of the new program is ec2-consistent-snapshot.

Over the last year as I and my co-workers have been using this code in production systems, we identified a number of ways it could be improved. Or, put another way, some serious issues came up when the idealistic world view of the original simplistic script met the complexities which can and do arise in the brutal real world.

A Scalable DNS scheme for Amazon’s EC2 Cloud

Monday, September 28th, 2009

One of the fundamental issues to deal with while building out my company’s cloud deployment was the need to assign our own hostnames to our instances. To accomplish this, we needed some sort of internal dns solution. One complication we encountered was that the instances where located in different EC2 regions (EU, US and soon, a possible second US region). Each Amazon region has it’s own private class A ip range, to which the instances are NAT’d, and public class B’s.