Jon Zobrist dot tumblr dot com

Here is my script that will backup files & MySQL db’s to Amazon S3, and restore them.

Thanks to the team over at Chatterfly, check them out for sponsoring this scripts development.

The use case I wrote this for was having an auto-scaling group that included the latest database. For infrequently changing LAMP sites like my Wordpress blog, which include a MySQL database, you run the script to update S3 after making changes. Then when traffic gets heavier, your auto-scaling group launches more instances, they boot up and download the latest files and db, and then join the ELB pool.

It also works well for backing up a non-EC2 Linux server to S3 for Disaster Recovery (DR).

WARNING: Running the start command on the wrapper script, or on the bluesun-setup.sh script will DELETE the local versions of everything the script otherwise backs up. I recommend NEVER adding this script to init on your MASTER server. If you only want backups, run the script only with updateS3. Then, when you need to restore, run it manually with start.

Steps to setup backups to Amazon S3 using this script.

1. Download script from github, here http://bit.ly/LnhMXb
2. Copy scripts somewhere on your server, I use /etc/init.d. Mark script bluesun-setup.sh executable.
3. Create directory /etc/bluesun-setup and put server.conf in it.
4. Edit /etc/bluesun-setup/server.conf. Read the comments, it is heavily commented and includes use cases.
5. Download and setup Tim Kay’s excellent Amazon AWS tools. Follow Tim’s instructions for setting up your AWS credentials file.
6. Setup a mysql user, and put the credentials in root’s my.cnf.
7. Create an Amazon S3 bucket to store your files.
8. Push your files to Amazon S3 manually.
9. Setup a cron job to push them however more frequently you like.

Steps to restore from Amazon S3 using this script.

1. Follow steps 1-6 from above.
2. Copy the /etc/bluesun-setup/server.conf from the original server to the restore server.
3. Restore from S3.

Detailed version of backup to Amazon S3 using this script

1. Download script from github, here http://bit.ly/LnhMXb
2. Copy scripts somewhere on your server, I use /etc/init.d. Mark script bluesun-setup.sh executable.
3. Create directory /etc/bluesun-setup and put server.conf in it.
4. Edit /etc/bluesun-setup/server.conf. Read the comments, it is heavily commented and includes use cases.
5. Download and setup Tim Kay’s excellent Amazon AWS tools. Follow Tim’s instructions for setting up your AWS credentials file.
6. Setup a mysql user, and put the credentials in root’s my.cnf.
7. Create an Amazon S3 bucket to store your files.
8. Push your files to Amazon S3 manually.
9. Setup a cron job to push them however more frequently you like.

cd /etc/init.d

wget http://bit.ly/LxKY9X

chmod uog+x bluesun-setup.sh

mkdir -p /etc/bluesun-setup

cd /etc/bluesun-setup

wget http://bit.ly/LxKVLw

#Edit server.conf. At a minimum set S3_BUCKET, DIRS and one of MYSQL_FILENAME or MYSQL_DATABASES.

mkdir -p /root/bin

cd /root/bin

curl http://bit.ly/LxKY9Z -o aws

chmod uog+x aws

perl aws --install

touch /root/.awssecret

chmod og-rwx /root/.awssecret

#Edit /root/.awssecret. Add your AWS credentials, put the ACCESS KEY ID on the first line, and the SECRET KEY on the second line

s3mkdir backups.example.com

touch /root/.my.cnf

chmod og-rwx /root/.my.cnf

#Create your mysql user, if not using root. A typical grant for a read only backup user could be

# mysql> GRANT SELECT, LOCK TABLES ON *.* TO user@'localhost' IDENTIFIED BY 'password';

#Edit /root/.my.cnf so it has a mysql [client] section that includes your username and password, so it looks like this:

[client]

user="user"
password="password"

#Now backup to your new S3 bucket with

/etc/init.d/bluesun-setup.sh updateS3

#Check that your files are there

s3ls backups.example.com

#Setup a cron job for more backups

crontab -e

#Add this line & save

0 * * * * /etc/init.d/bluesun-setup.sh updateS3

To restore follow all the same steps except manually run

/etc/init.d/bluesun-setup.sh start

Also, don’t setup that in a cron job

That should be it! Email me jon@jonzobrist.com or ping me on twitter @jonzobrist

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.