... more stuff
at
php-app-engine.com

Archive for the ‘Statistics’ Category

Scaling (Down) with AWS

Sunday, December 6th, 2009

http://wi.nr/4G

Launching a new webapp is never easy – even one as simple as a URL shortener. Will it catch on? If it does, what does that mean in terms of traffic? 10, 100, 1000 requests per second?

A few weeks ago we did some back of the envelope calculations for how big wi.nr could get in the best case. The calculation went something like: 10,000 active users x 10 shortened URLs per day x 100 people clicking on each of those URLs = 10M shortened URLs clicked a day or ~120 per sec.

Reddit on EC2

Tuesday, November 10th, 2009

http://www.reddit.com/r/IAmA/comments/a2zte/i_run_reddits_servers_and_do_a_bunch_of_other/

I made a blog post today about our move to the cloud, and thought I would give you all the chance to ask me questions, too. I’ll answer anything I can, and if I can’t, I’ll let you try to let you know.

To get the discussion going, here are some fun stats about our servers:

218 Virtual CPUs 380GB of RAM
9TB of Block Storage
2TB of S3 Storage
6.5 TB of Data Out / mo
2TB of Data In / mo
156M+ Pageviews

Amazon’s EC2 Generating 220M+ Annually

Thursday, October 1st, 2009

http://cloudscaling.com/blog/cloud-computing/amazons-ec2-generating-220m-annually

How Big is Amazon’s EC2?
Big. 40,000 servers. I have independently confirmed this with at least two sources close to EC2. Obviously, I can’t reveal them, but they are personally known to me and reliable. The first source gave me the 40,000 number and the second confirmed that the number is close. At most, we’re talking +/- 10,000 servers, so within 25%, but I’m guessing I’m very close. More like +/- 5K. Regardless, within 25% is more than close enough for us to get a pretty good gauge. For our purposes today we’ll go with the 40K number.

Anatomy of an Amazon EC2 Resource ID

Wednesday, September 30th, 2009

http://www.jackofallclouds.com/2009/09/anatomy-of-an-amazon-ec2-resource-id/

Each time you allocate a resource using EC2 – an instance, a volume or a snapshot – you receive a unique identifier. This is the EC2 resource ID. Have you ever wondered what this ID represents? Well, I did. After noticing similarities between the IDs of resources requested in close succession, I started digging.

http://blog.rightscale.com/2009/10/05/amazon-usage-estimates/

First of all, Guy’s analysis contains one significant error which is due to the limited data set he had access to. Before May 2009 EC2 issued even and odd instance IDs, not just even ones as he mentions. Since that date EC2 issued only even IDs until it switched to only odd ones in early September. The even/odd switches don’t seem to correlate with ID boundaries, perhaps Amazon switches between two active/standby reservation systems or something else is going on.

The formula to convert an EC2 ID into a sequential launch number as far as we call tell is:

Given an aws id as i-11223333
Assign p1 the 1’s, p2 the 2’s and p3 the 3’s
Also assign p31 the first two 3’s and p32 the last two 3’s
Compute:
c1 = (p1 ^ p32) ^ 0×69
c2 = (p2 ^ p31) ^ 0xe5
c3 = p3 ^ 0×4000
And finally concatenate c1-c2-c3. (This does not include the even/odd adjustments)

The upshot of Guy’s error is that he underestimates the launches by almost 2x!