Archiving and Compression

Archiving

Before proceeding further, you need to create some files and directories to backup and archive:

[root@centos ~]# mkdir -p /test/repY; mkdir /test/repZ
[root@centos ~]# cd /test/repY; touch Y1 Y2 Y3
[root@centos repY]# cd /test/repZ; touch Z1 Z2
[root@centos repZ]# ls -lR /test
/test:
total 8
drwxr-xr-x. 2 root root 4096 Oct 27 14:47 repY
drwxr-xr-x. 2 root root 4096 Oct 27 14:48 repZ
/test/repY:
total 0
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y1
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y2
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y3
/test/repZ:
total 0
-rw-r--r--. 1 root root 0 Oct 27 14:48 Z1
-rw-r--r--. 1 root root 0 Oct 27 14:48 Z2

tar

The tar command can be used to archive or back-up files to:

a special file such as a streamer,

an ordinary file on disk,

Standard Output to be used in a pipe.

tar is an abbreviation of tape archiver.

You can now proceed with the back-up of the directory test and it's contents to an ordinary file:

[root@centos repZ]# tar cvf /tmp/test.tar /test
tar: Removing leading `/' from member names
/test/
/test/repY/
/test/repY/Y3
/test/repY/Y2
/test/repY/Y1
/test/repZ/
/test/repZ/Z1
/test/repZ/Z2

To consult the archive's table of contents, you need to use the following command:

[root@centos repZ]# tar tvf /tmp/test.tar
drwxr-xr-x root/root 0 2013-10-27 14:47 test/
drwxr-xr-x root/root 0 2013-10-27 14:47 test/repY/
-rw-r--r-- root/root 0 2013-10-27 14:47 test/repY/Y3
-rw-r--r-- root/root 0 2013-10-27 14:47 test/repY/Y2
-rw-r--r-- root/root 0 2013-10-27 14:47 test/repY/Y1
drwxr-xr-x root/root 0 2013-10-27 14:48 test/repZ/
-rw-r--r-- root/root 0 2013-10-27 14:48 test/repZ/Z1
-rw-r--r-- root/root 0 2013-10-27 14:48 test/repZ/Z2

In order to create an incremental back-up, you now need to create an empty file to be used as a time reference file. All files modified or created after the creation of this file will be included in the incremental archive:

[root@centos repZ]# touch /tmp/dateref

Having created your reference file, you can now proceed with making some changes to some files:

[root@centos repZ]# echo "Red Hat is great \!" > /test/repY/Y1
[root@centos repZ]# echo "Red Hat is wonderful \!" > /test/repZ/Z1

Now you can test the validity of your reference file by using the find command:

[root@centos repZ]# find /test -newer /tmp/dateref > /tmp/tar.liste
[root@centos repZ]# cat /tmp/tar.liste
/test/repY/Y1
/test/repZ/Z1

Having tested /tmp/dateref you can now use this file to create an incremental archive using the -N option of the tar command:

[root@centos repZ]# tar -cvf /tmp/incremental.tar -N /tmp/dateref /test
tar: Removing leading `/' from member names
/test/
/test/repY/
tar: /test/repY/Y3: file is unchanged; not dumped
tar: /test/repY/Y2: file is unchanged; not dumped
/test/repY/Y1
/test/repZ/
/test/repZ/Z1
tar: /test/repZ/Z2: file is unchanged; not dumped

Check the contents of the archive by using the -t option of the tar command:

[root@centos repZ]# tar tvf /tmp/incremental.tar
drwxr-xr-x root/root 0 2013-10-27 14:47 test/
drwxr-xr-x root/root 0 2013-10-27 14:47 test/repY/
-rw-r--r-- root/root 20 2013-10-27 17:01 test/repY/Y1
drwxr-xr-x root/root 0 2013-10-27 14:48 test/repZ/
-rw-r--r-- root/root 24 2013-10-27 17:01 test/repZ/Z1

<note important> Note that the archive contains the directories test, repY and repZ in addition to the two modified files Y1 and Z1. </note>

Now delete the contents of the test directory:

[root@centos repZ]# rm -rf /test/*

To restore your first archive, you need to be at the root of your file system:

[root@centos repZ]# cd /
[root@centos /]# tar xvf /tmp/test.tar
test/
test/repY/
test/repY/Y3
test/repY/Y2
test/repY/Y1
test/repZ/
test/repZ/Z1
test/repZ/Z2

Using the ls command, you can check that the contents of the test directory have been restored:

[root@centos /]# ls -lR /test
/test:
total 8
drwxr-xr-x. 2 root root 4096 Oct 27 14:47 repY
drwxr-xr-x. 2 root root 4096 Oct 27 14:48 repZ
/test/repY:
total 0
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y1
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y2
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y3
/test/repZ:
total 0
-rw-r--r--. 1 root root 0 Oct 27 14:48 Z1
-rw-r--r--. 1 root root 0 Oct 27 14:48 Z2

<note important> Note that both Y1 and Z1 are empty. </note>

Now restore your incremental archive:

[root@centos /]# tar xvf /tmp/incremental.tar
test/
test/repY/
test/repY/Y1
test/repZ/
test/repZ/Z1

Using the ls command, you can check that the contents of incremental.tar have been restored:

[root@centos /]# ls -lR /test
/test:
total 8
drwxr-xr-x. 2 root root 4096 Oct 27 14:47 repY
drwxr-xr-x. 2 root root 4096 Oct 27 14:48 repZ
/test/repY:
total 4
-rw-r--r--. 1 root root 20 Oct 27 17:01 Y1
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y2
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y3
/test/repZ:
total 4
-rw-r--r--. 1 root root 24 Oct 27 17:01 Z1
-rw-r--r--. 1 root root 0 Oct 27 14:48 Z2

<note important> Note that both Y1 and Z1 have been restored. </note>

cpio

cpio stands for Copy Input To Output. As well as having its own archive format, cpio can also manage tar files.

Before using cpio, you need to construct a list of files to be included in the archive :

[root@centos /]# find /test > /tmp/cpio.list
[root@centos /]# cat /tmp/cpio.list
/test
/test/repY
/test/repY/Y3
/test/repY/Y2
/test/repY/Y1
/test/repZ
/test/repZ/Z1
/test/repZ/Z2

Using both the cpio command and the contents of the above file, an archive can be created using the following command:

[root@centos /]# cpio -ov < /tmp/cpio.list > /tmp/test.cpio
/test
/test/repY
/test/repY/Y3
/test/repY/Y2
/test/repY/Y1
/test/repZ
/test/repZ/Z1
/test/repZ/Z2
1 block

To consult the archive's table of contents, you need to use the following command:

[root@centos /]# cpio -it < /tmp/test.cpio
/test
/test/repY
/test/repY/Y3
/test/repY/Y2
/test/repY/Y1
/test/repZ
/test/repZ/Z1
/test/repZ/Z2
1 block

Now delete the contents of the test directory:

[root@centos /]# rm -rf /test/*

Restore the deleted files with the following command:

[root@centos /]# cpio -ivdum < /tmp/test.cpio
/test
/test/repY
/test/repY/Y3
/test/repY/Y2
/test/repY/Y1
/test/repZ
/test/repZ/Z1
/test/repZ/Z2
1 block

Using the ls command, you can check that the contents of incremental.tar have been restored:

[root@centos /]# ls -lR /test
/test:
total 8
drwxr-xr-x. 2 root root 4096 Oct 27 17:09 repY
drwxr-xr-x. 2 root root 4096 Oct 27 17:09 repZ
/test/repY:
total 4
-rw-r--r--. 1 root root 20 Oct 27 17:01 Y1
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y2
-rw-r--r--. 1 root root 0 Oct 27 14:47 Y3
/test/repZ:
total 4
-rw-r--r--. 1 root root 24 Oct 27 17:01 Z1
-rw-r--r--. 1 root root 0 Oct 27 14:48 Z2

dd

dd stands for Disk to Disk Copy and is not considered to be backup command.

dd copies the file presented on Standard Input to the file on Standard Output limiting the number of blocks copied by the use of two options:

count

the number of blocks.

bs

the size of each block.

The dd command is therefore very useful for backing up the MBR (Master Boot Record) and the FAT (File Allocation Table).

Use of the following command backs up the MBR to a file called mbr.save:

[root@centos /]# dd if=/dev/sda of=/tmp/mbr.save bs=1 count=446
446+0 records in
446+0 records out
446 bytes (446 B) copied, 0.0034581 s, 129 kB/s

Use of the following command backs up the FAT to a file called fat.save:

[root@centos /]# dd if=/dev/sda of=/tmp/fat.save bs=1 count=64 skip=446
64+0 records in
64+0 records out
64 bytes (64 B) copied, 0.000359079 s, 178 kB/s

dump et restore

dump and restore are commands that base their output on the format of the file system being backed up (ext2, ext3, ext4). Due to this fact, only complete file systems can be backed up with the dump command and not specific directories within a file system.

The file system must not be in use whilst being dumped. As a result it is advisable to unmount the file system prior to proceeding with a dump.

The dump command can manage 10 dump levels ranging from 0 to 9. Each time the file system is dumped, the dump level is specified and that information together with the dump date and time are saved to the file /etc/dumpdates.

By definition a dump 0 is a complete backup of the entire file system whilst dump 1 is an incremental backup.

dump stores all files and directories with relative paths. As a result, to restore a dump using the restore command you need to be positioned in the file system itself.

Compression

Before continuing, create an archive of the /test directory and it's contents using the tar command:

[root@centos /]# tar cvf /tmp/test.tar /test
tar: Removing leading `/' from member names
/test/
/test/repY/
/test/repY/Y3
/test/repY/Y2
/test/repY/Y1
/test/repZ/
/test/repZ/Z1
/test/repZ/Z2

Check and note the size of the archive:

[root@centos /]# cd /tmp; ls -l | grep test.tar
-rw-r--r--. 1 root root 10240 Oct 27 17:11 test.tar

gzip

Use the gzip command to compress the test.tar file:

[root@centos tmp]# gzip test.tar

Check and note the size of the tar.gz file:

[root@centos tmp]# cd /tmp; ls -l | grep test.tar.gz
-rw-r--r--. 1 root root 272 Oct 27 17:11 test.tar.gz

<note important> Note that the original test.tar file has disappeared. </note>

Uncompress the file using the gunzip command:

[root@centos tmp]# gunzip test.tar.gz

bzip2

Now use the bzip2 command to compress the test.tar file:

[root@centos tmp]# bzip2 test.tar

Check and note the size of the tar.bz2 file:

[root@centos tmp]# ls -l | grep test.tar.bz2
-rw-r--r--. 1 root root 268 Oct 27 17:11 test.tar.bz2

<note important> Note that the original test.tar file has disappeared. </note>

Uncompress the file using the bunzip2 command:

[root@centos tmp]# bunzip2 test.tar.bz2

<note> Examples of other commands used to compress files are zip, compress and rar. Use the manual system to see how each command works. </note>

~~DISCUSSION:off~~

<html> <center> Copyright © 2011-2014 Hugh Norris.<br><br> <a rel=“license” href=“http://creativecommons.org/licenses/by-nc-nd/3.0/”><img alt=“Creative Commons License” style=“border-width:0” src=“https://i.creativecommons.org/l/by-nc-nd/3.0/88x31.png” /></a><br />This work is licensed under a <a rel=“license” href=“http://creativecommons.org/licenses/by-nc-nd/3.0/”>Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License</a> </center> </html>