Due to recent software outtages, the Peninsula School District is investing heavily in the following practices. Documentation. VM snapshots. Docker commits. Testing environment. Intigration testing. Homegrown SAML SP (Service Provider) for testing/debugging.

VM snapshots

Being this is an Amazon EC2 server, our regular backup/snapshot tool Veeam does not offer a solution for backups. Amazon provides a command line interface for managing EC2 instances as well as other Amazon services like S3. I found a great bash script on Github by Colinbjohnson and others for automating EC2 snapshots. I have setup this script to run on a cronjob from the Amazon load balancer (VMAWSLDBLNC02).

The goal is the following:

  • Backup all running EC2 instances daily
  • Delete daily backups after 1 month
  • Keep 1 backup from the previous month for a month

I chose this server because it is running Amazon's proprietary image, and came with the Amazon CLI pre-installed. I configured this CLI to use the serv_aws_cli Amazon account.

[root@vmawswsldblnc02 ~]# aws configure

After configuring AWS CLI for our region and account, I cloned the script to the load balancer.

[root@vmawswsldblnc02 ~]# git clone https://github.com/colinbjohnson/aws-missing-tools

Then setup a cronjob to perform the nessisary opperations. I moved the nessisary script out of the root folder, and into the home of the ec2-user.

[root@vmawswsldblnc02 ~]# crontab -e

#Backup volumes daily, delete after 1 month
0 0 * * * ec2-user /home/ec2-user/ec2-automate-backup.sh -r us-west-2 -s tag -t "Backup,Values=true" -k 31  
#backup volumes once a month, delete after 2 months
0 0 1 * * ec2-user /home/ec2-user/ec2-automate-backup.sh -r us-west-2 -s tag -t "Backup,Values=true" -k 62

Take note, the ec3-automated-backup script NEEDS the region defined. Even though we define the region in the aws configuration, this script has no idea and will return errors like:

A client error (InvalidVolume.NotFound) occurred when calling the DescribeVolumes operation: The volume 'vol-b82f54a8' does not exist.  
An error occurred when running ec2-describe-volumes. The error returned is below:  

(Followed by a new line and nothing)

This script is setup to only backup volumes with the tag of Backups with a value of true

The Amazon Add/Edit tags interface should be configured as shown for the desired volumes.

Amazon Tag Configuration

If you now look at the Amazon snapshots, you will see the following tags

Automated Backups tags

... Work in Progress