PHP App Engine

http://code.google.com/p/redis-admin/

Redis Admin, or ReAdmin, is a open source web interface to the Administration of Redis. ReAdmin is fully written in PHP using Redis, of course.

http://rediska.geometria-lab.net/

Rediska (radish on russian) – PHP client for Redis.

Redis is an advanced fast key-value database written in C. It can be used like memcached, in front of a traditional database, or on its own thanks to the fact that the in-memory datasets are not volatile but instead persisted on disk. One of the cool features is that you can store not only strings, but lists and sets with atomic operations to push/pop elements.

http://neo4j.org/

You can think of Neo4j as a high-performance graph engine with all the features of a mature and robust database. The programmer works with an object-oriented, flexible network structure rather than with strict and static tables — yet enjoys all the benefits of a fully transactional, enterprise-strength database.

Neo4j has been in commercial development for 8 years and in production for over 5 years. It is a mature and robust graph database that provides:

http://www.fossil-scm.org/index.html/doc/tip/www/theory1.wiki

Two questions (or criticisms) that arise frequently regarding Fossil can be summarized as follows:

Neither question can be answered directly because they are both based on false assumptions. We claim that Fossil is not based on SQLite at all and that Fossil is not based on a distributed NoSQL database because Fossil is a distributed NoSQL database. And, Fossil does use a modern high-level language for its implementation, namely SQL.

http://www.ipconfig.co.nz/blog/post.cfm/s3-bash

s3-bash is a small collection of BASH scripts to let you use it from an Unix, Linux or Mac OS X command line without needing Perl, Python, Java, .Net, etc…

http://getcloudfusion.com/

* Fast, powerful PHP toolkit with an easy to learn, consistent API.
* Highly extensible framework for easily adding new services.
* Ridiculously thorough API reference, code samples and tutorials.
* Works with Amazon Web Services and Eucalyptus.

Used to be called ‘Tarzan’ before.
Nice move with Eucaliptus.
Nice move with the name too.

http://news.ycombinator.com/item?id=705977

scalien.com

Keyspace

This paper describes PaxosLease, a distributed algorithm for lease negotiation. PaxosLease is based on Paxos, but does not require disk writes and does not make clock synchrony and skew assumptions. PaxosLease is used for master lease negotation in the open-source Keyspace replicated key-value store.

http://code.google.com/p/mcollective/

The Marionette Collective aka. mcollective is a framework to build server orchestration or parallel job execution systems.

We’ve attempted to think out of the box a bit designing this system by not relying on central inventories and tools like SSH, we’re not simply a fancy SSH “for loop”. MCollective uses modern tools like Publish Subscribe Middleware and modern philosophies like real time discovery of network resources using meta data and not hostnames. Delivering a very scalable and very fast parallel execution environment.

http://code.google.com/p/jclouds/

jclouds is an open source framework that helps you get started in the cloud and reuse your java development skills. Our api allows you to freedom to use portable abstractions or cloud-specific features. We support many clouds including Amazon, VMWare, Azure, and Rackspace.

http://mapbox.com/

MapBox is a suite of open source tools to create beautiful custom maps in Amazon’s cloud.

http://code.google.com/p/pigpy/

pypig – a python tool to manage Pig reports

pypig is an attempt to fill in these holes by providing a python module that knows how to talk to a Hadoop cluster and can create and manage complex report structures.

http://code.google.com/p/appscale/

AppScale is a platform that allows users to deploy and host their own Google App Engine applications. It executes automatically over Amazon EC2 and Eucalyptus as well as Xen and KVM. It has been developed and is maintained by the RACELab at UC Santa Barbara.

http://mrflip.github.com/wukong/index.html

Wukong: Hadoop made so easy a Chimpanzee could run it.

Send Wukong questions to the Infinite Monkeywrench mailing list

http://rubyhitsquad.com/Ruby_Hit_Squad.html

Basically, attack on Capistrano.
Grab the popcorn.

• Do the simplest thing that could possibly work.
• Nothing to 1.0 in four(ish) days.
• Targets the 80% use case.
• Uses Rake, as god intended.
• Use the right tool for the job (ssh, rsync, etc).
• Fold in the Rails Machine recipes.
• Clever is bad. Period.

http://developer.amazonwebservices.com/connect/entry.jspa?%20externalID=128

Curl is a popular command-line tool for interacting with HTTP services. This Perl script calculates the proper signature, then calls Curl with the appropriate arguments.

http://web.mit.edu/stardev/cluster/

Multiple Clusters – Currently, StarCluster only supports launching a single cluster on ec2. In theory, the software should be able to START multiple clusters but it’s not equipped to handle the accounting on which nodes belong to which cluster after the initial startup. This means things like listing the nodes, terminating a particular cluster, etc will not work. Support for the correct account of multiple clusters should come in future versions.

Dynamic Load Balancing – Support for a dynamically resizing cluster on ec2. Integrating the Service Domain Manager (SDM) and Hedeby software products from SUN into the AMI will allow ec2 nodes to easily be added to the Sun Grid Engine queue. This means you could theoretically start a single node cluster and as the load increases, ec2 nodes would be launched, added to the cluster, used for computation, and then removed when they’re idle. The impact of this would be to significantly lower the cost of using EC2 by only having one node up 24/7 and adding/removing nodes as needed.

http://www.cascading.org/

Cascading is a feature rich API for defining and executing complex, scale-free, and fault tolerant data processing workflows on a Hadoop cluster.

Cascading is Open Source and dual licensed under the GPL and OEM/Commercial Licenses. OEM/Commercial Licenses and Developer Support can be obtained through Concurrent, Inc.

CI in a box and Hudson.

http://thediscoblog.com/2009/11/24/provisioning-a-hudson-ci-server/

http://www.ciinabox.com/

CI in a Box is one of the easiest ways to get up and running with Continuous Integration– in fact, if you don’t believe me, check out the CI in a Box tutorial video. As you’ll see, CI in a Box makes setting up a Hudson CI server practically a breeze by leveraging Amazon’s EC2; what’s more, the video quickly sets up an SVN project that contains an Ant build (don’t worry, CI in a Box supports Maven as well!).

http://incubator.apache.org/projects/trafficserver.html

Traffic Server fills the need for a fast, extensible and scalable HTTP 1.1 proxy and cache. We have a production proven piece of software that can deliver HTTP traffic at high rates, and can scale well on modern SMP hardware. We have benchmarked Traffic Server to handle in excess of 35,000 RPS on a single box. Traffic Server has a rich feature set, implementing most of HTTP/1.1 to the RFC specifications.

http://workspace.globus.org/

Nimbus is an open source toolkit that allows you to turn your cluster into an Infrastructure-as-a-Service (IaaS) cloud. Feature highlights include:

Implementation based on the Xen hypervisor (KVM coming soon), read more about supported virtualization technologies…

Wow. Build you own EC2.
And try selling it. Ra!

http://aws.28msec.com/

We are proud to announce the second beta release of the libaws project (version 0.9.2) Libaws is an easy-to-use code library that helps you programming C++-Software that communicates with Amazon Web Services. More in detail, it supports communication with the following amazon services:

* Amazon Simple Storage Service (Amazon S3)
* Amazon SimpleDB
* Amazon Simple Queue Service (Amazon SQS)

http://code.google.com/p/cloudmapreduce/

Cloud MapReduce was developed at Accenture Technology Labs by Huan Liu and Dan Orban. It is a MapReduce implementation on top of the Amazon Cloud OS.

See Command line options for details on how to specify a job run, and Pre-built AMI for how to use the pre-built AMI image to make running the job easier. A tutorial is coming soon.

http://github.com/truthtrap/ec2-elastic-backups

i put together these 2 files because i wanted dirvish like backups, but with snapshots and not too much work. all in all
it took me still to much, but thanks to tools like simpledb (http://code.google.com/p/amazon-simpledb-cli/) and the
wonderful amazon aws tools i have what i want.

http://incubator.apache.org/cassandra/

Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store. Cassandra brings together the distributed systems technologies from Dynamo and the data model from Google’s BigTable. Like Dynamo, Cassandra is eventually consistent. Like BigTable, Cassandra provides a ColumnFamily-based data model richer than typical key/value systems.

Cassandra was open sourced by Facebook in 2008, where it was designed by one of the authors of Amazon’s Dynamo. In a lot of ways you can think of Cassandra as Dynamo 2.0. Cassandra is in production use at Facebook but is still under heavy development.

http://wiki.github.com/kirktrue/voldemort/ec2-testing-infrastructure

An open source clone of Amazon’s Dynamo

Goals and Deliverables

1. Ability to initialize EC2 instances for use with Voldemort
2. Cluster deployment, configuration
3. Individual node start/stop
4. Ability to leverage above for performance and correctness tests

http://4store.org/

4store, an efficient, scalable and stable RDF database

When configured as a cluster, import performances of 120 kT/s are easily achievable. Query times for relatively simple queries are often in the low milliseconds, even over the standard HTTP SPARQL protocol.

http://thinklinks.wordpress.com/2009/10/27/4store-amazon-machine-image-and-billion-triple-challenge-data-set/

Today, we are making publicly available an Amazon Machine Image for 4store. Additionally, we are making an Elastic Block Storage snapshot of the BTC dataset for 4store. Thus, developers can easily get started using 4store with a billion triples on Amazon’s cloud.

http://www.mongodb.org/display/DOCS/Home

The best features of document databases, key/values stores, and RDBMSes in one.

http://www.mongodb.org/display/DOCS/Amazon+EC2

MongoDB runs well on Amazon EC2 . This page includes some notes in this regard.

http://timkay.com/aws/

aws is a command-line tool that gives you easy access to Amazon EC2 and Amazon S3. aws is designed to be simple to install and simple to use.

Thanks to your feedback, aws is the top-rated “community code” for all of Amazon EC2 and S3! See the ratings and reviews at EC2 and S3. They make me blush! Thank you!

http://timkay.com/tools/
many other tools are kewl

http://github.com/adamwiggins/sumo/

Tired of wrestling with server provisioning? Sumo!
Want to fire up a one-off EC2 instance, pronto? ec2-run-instances got you down? Try Sumo.

http://ec2onrails.rubyforge.org/

EC2 on Rails is an Ubuntu Linux server image for Amazon’s EC2 hosting service that’s ready to run a standard Ruby on Rails application with little or no customization. It’s a Ruby on Rails virtual appliance.