tar: Incremental Dumps
1
1 5.2 Using 'tar' to Perform Incremental Dumps
1 ============================================
1
1 "Incremental backup" is a special form of GNU 'tar' archive that stores
1 additional metadata so that exact state of the file system can be
1 restored when extracting the archive.
1
1 GNU 'tar' currently offers two options for handling incremental
1 backups: '--listed-incremental=SNAPSHOT-FILE' ('-g SNAPSHOT-FILE') and
1 '--incremental' ('-G').
1
1 The option '--listed-incremental' instructs tar to operate on an
1 incremental archive with additional metadata stored in a standalone
1 file, called a "snapshot file". The purpose of this file is to help
1 determine which files have been changed, added or deleted since the last
1 backup, so that the next incremental backup will contain only modified
1 files. The name of the snapshot file is given as an argument to the
1 option:
1
1 '--listed-incremental=FILE'
1 '-g FILE'
1 Handle incremental backups with snapshot data in FILE.
1
1 To create an incremental backup, you would use '--listed-incremental'
1 together with '--create' (⇒create). For example:
1
1 $ tar --create \
1 --file=archive.1.tar \
1 --listed-incremental=/var/log/usr.snar \
1 /usr
1
1 This will create in 'archive.1.tar' an incremental backup of the
1 '/usr' file system, storing additional metadata in the file
1 '/var/log/usr.snar'. If this file does not exist, it will be created.
1 The created archive will then be a "level 0 backup"; please see the next
1 section for more on backup levels.
1
1 Otherwise, if the file '/var/log/usr.snar' exists, it determines
1 which files are modified. In this case only these files will be stored
1 in the archive. Suppose, for example, that after running the above
1 command, you delete file '/usr/doc/old' and create directory
1 '/usr/local/db' with the following contents:
1
1 $ ls /usr/local/db
1 /usr/local/db/data
1 /usr/local/db/index
1
1 Some time later you create another incremental backup. You will then
1 see:
1
1 $ tar --create \
1 --file=archive.2.tar \
1 --listed-incremental=/var/log/usr.snar \
1 /usr
1 tar: usr/local/db: Directory is new
1 usr/local/db/
1 usr/local/db/data
1 usr/local/db/index
1
1 The created archive 'archive.2.tar' will contain only these three
1 members. This archive is called a "level 1 backup". Notice that
1 '/var/log/usr.snar' will be updated with the new data, so if you plan to
1 create more 'level 1' backups, it is necessary to create a working copy
1 of the snapshot file before running 'tar'. The above example will then
1 be modified as follows:
1
1 $ cp /var/log/usr.snar /var/log/usr.snar-1
1 $ tar --create \
1 --file=archive.2.tar \
1 --listed-incremental=/var/log/usr.snar-1 \
1 /usr
1
1 You can force 'level 0' backups either by removing the snapshot file
1 before running 'tar', or by supplying the '--level=0' option, e.g.:
1
1 $ tar --create \
1 --file=archive.2.tar \
1 --listed-incremental=/var/log/usr.snar-0 \
1 --level=0 \
1 /usr
1
1 Incremental dumps depend crucially on time stamps, so the results are
1 unreliable if you modify a file's time stamps during dumping (e.g., with
1 the '--atime-preserve=replace' option), or if you set the clock
1 backwards.
1
1 Metadata stored in snapshot files include device numbers, which,
1 obviously are supposed to be non-volatile values. However, it turns out
1 that NFS devices have undependable values when an automounter gets in
1 the picture. This can lead to a great deal of spurious redumping in
1 incremental dumps, so it is somewhat useless to compare two NFS devices
1 numbers over time. The solution implemented currently is to consider
1 all NFS devices as being equal when it comes to comparing directories;
1 this is fairly gross, but there does not seem to be a better way to go.
1
1 Apart from using NFS, there are a number of cases where relying on
1 device numbers can cause spurious redumping of unmodified files. For
1 example, this occurs when archiving LVM snapshot volumes. To avoid
1 this, use '--no-check-device' option:
1
1 '--no-check-device'
1 Do not rely on device numbers when preparing a list of changed
1 files for an incremental dump.
1
1 '--check-device'
1 Use device numbers when preparing a list of changed files for an
1 incremental dump. This is the default behavior. The purpose of
1 this option is to undo the effect of the '--no-check-device' if it
11 was given in 'TAR_OPTIONS' environment variable (⇒
TAR_OPTIONS).
1
1 There is also another way to cope with changing device numbers. It
1 is described in detail in ⇒Fixing Snapshot Files.
1
1 Note that incremental archives use 'tar' extensions and may not be
1 readable by non-GNU versions of the 'tar' program.
1
1 To extract from the incremental dumps, use '--listed-incremental'
1 together with '--extract' option (⇒extracting files). In this
1 case, 'tar' does not need to access snapshot file, since all the data
1 necessary for extraction are stored in the archive itself. So, when
1 extracting, you can give whatever argument to '--listed-incremental',
1 the usual practice is to use '--listed-incremental=/dev/null'.
1 Alternatively, you can use '--incremental', which needs no arguments.
1 In general, '--incremental' ('-G') can be used as a shortcut for
1 '--listed-incremental' when listing or extracting incremental backups
1 (for more information regarding this option, ⇒incremental-op).
1
1 When extracting from the incremental backup GNU 'tar' attempts to
1 restore the exact state the file system had when the archive was
1 created. In particular, it will _delete_ those files in the file system
1 that did not exist in their directories when the archive was created.
1 If you have created several levels of incremental files, then in order
1 to restore the exact contents the file system had when the last level
1 was created, you will need to restore from all backups in turn.
1 Continuing our example, to restore the state of '/usr' file system, one
1 would do(1):
1
1 $ tar --extract \
1 --listed-incremental=/dev/null \
1 --file archive.1.tar
1 $ tar --extract \
1 --listed-incremental=/dev/null \
1 --file archive.2.tar
1
11 To list the contents of an incremental archive, use '--list' (⇒
list), as usual. To obtain more information about the archive, use
1 '--listed-incremental' or '--incremental' combined with two '--verbose'
1 options(2):
1
1 tar --list --incremental --verbose --verbose --file archive.tar
1
1 This command will print, for each directory in the archive, the list
1 of files in that directory at the time the archive was created. This
1 information is put out in a format which is both human-readable and
1 unambiguous for a program: each file name is printed as
1
1 X FILE
1
1 where X is a letter describing the status of the file: 'Y' if the file
1 is present in the archive, 'N' if the file is not included in the
1 archive, or a 'D' if the file is a directory (and is included in the
1 archive). ⇒Dumpdir, for the detailed description of dumpdirs and
1 status codes. Each such line is terminated by a newline character. The
1 last line is followed by an additional newline to indicate the end of
1 the data.
1
1 The option '--incremental' ('-G') gives the same behavior as
1 '--listed-incremental' when used with '--list' and '--extract' options.
1 When used with '--create' option, it creates an incremental archive
1 without creating snapshot file. Thus, it is impossible to create
1 several levels of incremental backups with '--incremental' option.
1
1 ---------- Footnotes ----------
1
1 (1) Notice, that since both archives were created without '-P' option
1 (⇒absolute), these commands should be run from the root file
1 system.
1
1 (2) Two '--verbose' options were selected to avoid breaking usual
1 verbose listing output ('--list --verbose') when using in scripts.
1
1 Versions of GNU 'tar' up to 1.15.1 used to dump verbatim binary
1 contents of the DUMPDIR header (with terminating nulls) when
1 '--incremental' or '--listed-incremental' option was given, no matter
1 what the verbosity level. This behavior, and, especially, the binary
1 output it produced were considered inconvenient and were changed in
1 version 1.16.
1