I just had a thought about the backup scripts out there. Why do we use .tar.gz or .tar.bz2? They work more or less, but I've found two problems with them:

- if you want to retrieve a single file from it, it's very inefficient since the only way is to sequentially scan the tar file for it
- if you have a 2.2 Linux kernel and the standard filesystem, your backup file will get truncated if it exceeds 2 GB

.tar makes sense when you're backing up to tape. But when you're backing up to disk, wouldn't it be better to make a copy of the entire directory tree, a la "cp -r", and then gzip individual files that can be compressed (e.g. not GIF, JPG, MP3, GZ, BZ2, ZIP, etc.)? This would make it a lot easier to retrieve files from the backup, and also solves the 2 GB file size limit problem.

Does anyone know of an incremental backup script that uses the technique I mentioned, or am I going to have to look into writing my own?

Compressing single tar file will give you a much smaller backup size then compressing single individual files.
This usualy works best when you are backuping thousands of small html files ...

Use rsync locally or rsync over ssh.

Try looking up rsync, from what I understand it can do incremental backups.

i think you'll need to write your own, pmak0
hehe...

rsync is indeed a nice tool, but doesn't do what you want (from what I understood, generating incremental backups of some compressed files and others not)
it just generates from uncompressed files

you had a good idea IMO

rsync can indeed compress files. man rsync or rsync -h for more details.

its linux format

Originally posted by doman
its linux format

Hmm. Rsync runs fine on Windows and it's gzip format, not Linux format. Gzip runs on Windows too.

fyi, the original poster specified Linux anyway.