... more stuff
at
php-app-engine.com

Archive for the ‘Backup’ Category

Tarsnap

Thursday, December 24th, 2009

http://www.tarsnap.com/

Online backups for the truly paranoid

Tarsnap is a secure online backup service for BSD, Linux, OS X, Solaris, Cygwin, and can probably be compiled on many other UNIX-like operating systems. The tarsnap client code provides a flexible and powerful command-line interface which can be used directly or via shell scripts.

At the present time, tarsnap does not support Windows (except via Cygwin) and does not have a graphical user interface.

Amazon EC2 backup scripting

Wednesday, December 16th, 2009

http://www.jayhaabee.nl/blog/?p=43

For my first useful experiment with Amazon Elastic Cloud Computing (EC2 for short), I choose remote backup. Simply put, we’ll write a simple script, that fires up an Amazon EC2 instance, attaches a storage device and sync my valuable data to the Amazon EC2 device. Oh, and lets not forget to shutdown the EC2 instance to save us some money.

How-To: Automated incremental daily backups to Amazon S3 using Duplicity

Sunday, December 13th, 2009

http://www.cenolan.com/2008/12/how-to-incremental-daily-backups-amazon-s3-duplicity/#more-54

This guide shows how to use Amazon S3 with duplicity to make secure GPG encrypted automated daily incremental backups (snapshots) of a Linux server or desktop. I have been using this method on various servers for several months and it has proved to be a reliable, secure, cheap, and robust method to create automated backups.

I have used this method on Fedora, YDL, and CentOS but the instructions should equally apply to other Linux distributions including Debian and Ubuntu. It will even work on OS X using the MacPorts version of duplicity.

ec2-elastic-backups

Tuesday, November 10th, 2009

http://github.com/truthtrap/ec2-elastic-backups

i put together these 2 files because i wanted dirvish like backups, but with snapshots and not too much work. all in all
it took me still to much, but thanks to tools like simpledb (http://code.google.com/p/amazon-simpledb-cli/) and the
wonderful amazon aws tools i have what i want.

Automating EC2 EBS Snapshot Cleanup

Monday, November 2nd, 2009

http://www.techkismet.com/systems-admin/automating-ec2-ebs-snapshot-cleanup.html

So the problem in a nutshell is I have 10 volumes, each of which is cron’ed to be snapshotted at various times of the day (depends on the specific volume as to how often it is backed up). With 10 volumes, my S3 storage costs can get out of hand quite quickly. So I needed to develop a set of scripts that would scan my snapshots – and remove the oldest ones – so I’m not paying for that storage.

Rsync to Amazon S3

Sunday, October 25th, 2009

http://www.s3rsync.com/index.php/Rsync_to_Amazon_S3

The problem:
S3 storage protocol is “all or nothing” meaning that you can not modify a file on S3 and even for a minor change you have to upload (S3 PUT) the whole file again. This implies that backup and synchronization to S3 are inefficient. Any file modification forces you to upload the whole new file again, not just what was changed. So you are wasting bandwidth and the backup process is significantly slower!

Our solution:
S3rsync resolves this limitation and allows you to use Rsync bandwidth efficient algorithm that enables to upload only partial files that were changed. This is done by connecting to our Rsync servers located inside Amazon facility (Ec2). Using this functionality enables you to fully benefit from Rsync power.

Automatic Elastic Block snapshots with a cron job

Wednesday, October 21st, 2009

http://www.floydprice.com/2009/10/automatic-elastic-block-snapshots-with-a-cron-job/


Amazon EC2 really is amazing, and the Elastic Block storage is pretty darn good too, however I wish you could automate the snapshot process form the EC2 console.

It is however pretty easy to do yourself using a simple cron job.

Before you start make sure you have a JRE:

MySQL backups with EBS snapshots

Sunday, October 18th, 2009

http://blog.labnotes.org/2009/10/13/mysql-backups-with-ebs-snapshots/

Amazon EBS snapshots are awesome. You can take snapshots to backup your database, quickly move data between staging and production, run analytics on the latest data, resize volumes and much more.

For Apartly, I’m using MySQL, storing data and log files on a mounted EBS volume. EBS offers persistent, reliable storage (with replication for fault tolerance). Still, nothing can prevent data from disappearing, hence backups.

Cloud Tips: Automatic Backups to S3

Thursday, October 15th, 2009

http://kovshenin.com/archives/cloud-tips-automatic-backups-to-s3/

In a previous post about backing up EC2 MySQL to an Amazon S3 bucket we covered dumping MySQL datasets, compressing them and uploading to S3. After a few weeks test-driving the shell script, I came up with a new version that checks, fixes and optimizes all tables before generating the dump. This is pretty important as mysqldump will fail on whatever step would cause an error (data corruption, crashed tables, etc), thus your uploaded to S3 archive would be kind of corrupt. Here’s the script: