Note on using webdav idisk for experiment data

One of the problems I had before with setting up a script to manage the experimental scenario and data on a webdav idisk was that I hadn't stumbled across how to automatically mount & unmount the filesystem. It turns out that this is very easy.

First, create a directory to use as the mountpoint. For example, assuming you are in a writable directory, use something like this:

mkdir mnt

Next, use a command of this form:

/sbin/mount_webdav -s http://idisk.mac.com/groups.labname mnt

("groups.labname" should be replaced with the actual name of your idisk)

If the login info us not in your keychain, it will ask you for it. You might consider putting in the keychain for convenience. Note that any user (i.e., any RA who will access the database) must have access to the idisk.

Note that it is possible for a subdirectory to mounted directly with a command like this:

/sbin/mount_webdav -s http://idisk.mac.com/groups.labname/Databases/Thisdatabase mnt

However, in this case, a separate keychain entry will be needed to log in. It would normally be simpler to mount the root of the idisk and navigate its hierarchy programatically.

When complete, a simple "umount mnt" will be required.

Note that this method will work even if the idisk had already been mounted in the standard location or somewhere else.

Combining image and rsync backups

This is from a note on the Apple server list.

The procedure is to start out with an asr image of the root volume using the -erase flag. This is now a bootable volume.

Then, each night, perform an rsync on this volume, using -delete and other flags so that all changes are written to the volume. This is now an updated, but still fully bootable volume.

This is almost equivalent to doing a nightly clone. It is food for thought. Here are some related issues:

• Unless you want to bring the system down to single user mode every night, you are still going to have to deal with open database files during the rsync.
  
• Assuming that those files have been dealth with, would it be faster just to do the image dump every night?
• Is the target system more likely to be bootable in the even of a catastrophe mid-copy if rsync is used?
• One of the biggest advantages of using rsync is the ability to do snapshots. Clearly, the original image could be used as the starting point for nightly snapshots, but then the image itself would never be updated. Is there some way for the root hierarchy to be the most recent snapshot, with previous versions stored elsewhere? (See below)

It seems to me that what might be needed here is for there to be two rsyncs each night. The first one is to update the bootable image from the working image, as in the original hint. The second one, to be done when the first one is finished, is to make a snapshot of the bootable image.

In other words, the backup would go like this:

1. Before any backup, go through the process of dumping all system databases.
   
2. For the first (labor intensive) backup, use asr to make a complete copy of the base system. The system should be as quiesent as possible for this, and the system database dumps should contain the current database contents. The backup volume should be considerably larger than the working volume.
3. Dump the databases and use rsync to update the smaller backup volume. Be careful to exclude the /snapshots directory on the backup volume; this should be untouched by this run of rsync.
4. Then use rsync again to create a new snapshot of the non-snapshot regions of the backup volume (there is no need for an additional database dump). This will be stored in /snapshots on that volume (and so /snapshots will again be excluded).

Both runs of rsync could be run at fairly low priorities.

Note that if this were to be done over the network via an rsync server, the second (snapshot) rsync run could be done locally on the server. Not sure if that would be worth it, though. For now, I'm assuming an attached drive.

Also, it would be possible for /snapshots to be a different drive. For example, with two internal drives, the second drive could be the mirror of the first, with a larger external firewire drive for the snapshots. In a fairly non-intensive application like in our lab, this use of the second internal drive would probably be better than mirroring RAID, which is how it is being used now.

In any case, we would want to avoid spotlight on the backup drives, and also we would want them to be mounted read-only except when actually being written during backups.

A variation of the above would be to split drive 1 between the backup-able part and a large non-backupable part, so that less space would be needed on drive 2.