====== Working with Disk Images in Linux ======

See also: **[[computing:backup:backup_cf_ssh|Backing Up Your Compact Flash Using SSH]]**

===== RSYNC To Download ISO Images =====

Use rsync to download large files for efficient use of time and bandwidth. Rsync only transfers the changed portions of files. Rsync servers are widely available, but not as available as FTP.

==== Check an rsync server for the file you want ====

:!: Remember the trailing backslash.

<file>
rsync rsync://distro.ibiblio.org/smeserver/releases/testing/8.0beta7/iso/i386/
</file>

==== Download a file ====

Rename the most similar file. Only the differences in this file and the file to be downloaded will actually be transferred. This example downloads an SME Server install CD image.

At the Linux command line:

<file>
cd destination/dir
mv smeserver-8.0beta6-i386.iso smeserver-i386.iso
</file>

:!: Note that the dot at the end is important!

Start the download:

<file>
rsync -avv --stats distro.ibiblio.org::smeserver/releases/testing/8.0beta7/iso/i386/smeserver-i386.iso .
</file>

or

<file>
rsync -avv --stats rsync://mirrors.login.com/centos/6/isos/x86_64/CentOS-6.0-x86_64-bin-DVD1.iso .
</file>

===== View Disk Partitions =====

To view disk and USB pen drive partitions, log in as root and enter.

<file>
fdisk -l
df
</file>

IDE hard drives and CD-ROM drives are accessed as /dev/hda (primary master) through /dev/hdd (secondary slave), or beyond depending on the IDE interfaces actually installed.

SCSI drives and USB storage devices are accessed as /dev/sda, /dev/sdb, etc.
Floppy Images

===== Create an image of a floppy disk =====

<file>
dd if=/dev/fd0 of=/path/to/file.img bs=1440k
</file>

Create a floppy from an image file:

<file>
dd if=/path/to/file.img of=/dev/fd0 bs=1440k
</file>

===== CD/DVD (ISO) Images =====

To mount an ISO image

<file>
mount -o loop -t iso9660 example.iso /mnt/cdrom
</file>

To make an ISO from your CD/DVD, place the media in your drive but do not mount it. If it automounts, unmount it.

For DVD:

<file>
dd if=/dev/dvd of=dvdname.iso
</file>

For CD-ROM:

<file>
dd if=/dev/cdrom of=cdname.iso
</file>

If CD-ROM drive is SCSI:

<file>
dd if=/dev/scd0 of=cdname.iso
</file>

===== To make an ISO image from files on your hard drive =====

Create a directory which holds the files you want. Then use the mkisofs command.

<file>
mkisofs -o /tmp/cdname.iso /some/directory/
</file>

This results in a file called ''cd.iso'' in folder ''/tmp'' which contains all the files and directories in ''/some/directory/''.

===== To verify an ISO image file to a CD disk =====

Use the md5sum command. Note that a difference in the md5sums does NOT guarantee a failed burn, but if the md5sums are the same, it is most probably a good disk/image. To improve your chances of a successful disk-to-image comparison, be sure to fully erase CD/RW disks (not 'quick') and burn using 'disk at once' (DAO).

<file>
md5sum /dev/cdrom cdrom.iso
367635a8715713e3eb61e6859b76e2b2 /dev/cdrom
367635a8715713e3eb61e6859b76e2b2 cdrom.iso
</file>

If this simple test fails, you would need to use this more thorough test to verify the disk/image:

<file>
mount -o ro,loop -t iso9660 `pwd`/cdrom.iso /mnt
mount -o ro -t iso9660 /dev/cdrom
diff -r /media/cdrom /mnt
</file>

For more info, see the man pages for mkisofs, losetup, and dd, or see the CD-Writing-HOWTO at http://www.tldp.org.

===== DVD (ISO) Images =====

You can back up a DVD (even a movie) to your hard drive and restore it easily from the Linux command line.

Assuming your DVD drive is ''/dev/sr0'', execute the following from the command line:

<file>
dd if=/dev/sr0 of=dvd.iso
</file>

Verify that the DVD and the ISO file are exactly the same:

<file>
diff /dev/sr0 dvd.iso -r
</file>

Insert a blank DVD of the same type as the original (-R, +R, DVD5, DVD9).

Burn the new DVD from the ISO:

<file>
/usr/bin/growisofs -dvd-compat -Z /dev/dvd=dvd.iso
</file>

Erase a rewriteable DVD from the command line lke this:

<file>
dvdrecord dev=/dev/dvd -v blank=fast
</file>

===== Compact Flash or USB Key =====

==== Unmount the Filesystem ====

Many newer linux distributions will automatically mount flash media once it's been inserted. You may have to unmount the media first before imaging. If you see the flash card mounted on the desktop GUI, try right clicking on the icon and selecting "umount".

Alternately at the command prompt, show the mounted filesystems and unmount like this:

<file>
mount

umount /dev/sdxy
</file>

where "x" and "y" are lower-case letters of the device and partition to unmount. 

You can usually determine the proper device name to use with ''fdisk -l'' or ''dmesg''. You probably need to be root (or use ''sudo'') for these commands.

==== Backup an Existing Device ====

This command assumes you have a flash device ''/dev/sdf'' and you are logged in as a regular user, not root:

<file>
sudo sh -c "dd if=/dev/sdf | gzip > Desktop/8GBUSB.backup.img.gz"
</file>

==== Restore the Device ====

You might issue the following command as root to write an image onto the CF or USB key:

<file>
gunzip -c /path/to/imagefile.img.gz > /dev/sdf
</file>

If you want to do the same thing from a non-root account using sudo, try this format

<file>
sudo sh -c "gunzip -c /path/to/imagefile.img.gz > /dev/sdf"
</file>

Warning, if you are not sure where your CF device is, you could easily overwrite all data on another disk drive in your system!

===== DD Rescue =====

DD Rescue is a tool for disk data recovery. More info is [[computing:storage:data_recovery|here]].

===== DD Command =====

The dd command copies data from one place to another. Sometimes cat can do the same thing (with redirection), but dd has options to translate data, selectively copy only part of a data stream, and buffer its reads and writes.

Dd can copy a CD to an ISO file, copy one partition to another, or restore an image file to a disk. Using the count and seek options, an individual sector of a disk can be extracted without having to wait for the entire rest of the disk to be read.

Also see the very useful tool dd_rescue, which you can use to recover media with errors on it.

Since dd can be easily used to munge a hard drive, it (hopefully) is a superuser-only tool.

==== Examples ====

=== Creating and Restoring a hard drive backup image ===

The main options to be concerned about are if= (input file) and of= (output file). By default, dd reads from stdin and writes to stdout. Here is an example of a use for dd:

<file>
dd if=/dev/hda | gzip > /mnt/hdb1/system_drive_backup.img.gz
</file>

Here dd is making an image of the first hard drive, and piping it through the gzip compression program. The compressed image is then placed in a file on a separate drive. To reverse the process:

<file>
gzip -dc /mnt/hdb1/system_drive_backup.img.gz | dd of=/dev/hda
</file>

Here, gzip is decompressing (the -d switch) the file, sending the results to stdout (the -c switch), which are piped to dd, and then written to /dev/hda.
Create and Restore a Floppy Image

<file>
dd if=/dev/fd0 of=/tmp/floppy.img bs=10240
</file>

That will copy the contents of the floppy to a file. Then, to put the image onto a new floppy, swap "if" and "of" params.

<file>
dd if=/tmp/floppy.img of=/dev/fd0 bs=10240
</file>

===== Backing up your Master Boot Record (MBR) =====

You should do this before you edit your partition table so that you can put it back if you mess things up.

<file>
dd if=/dev/hda of=/root/hda.boot.mbr bs=512 count=1
</file>

If things get messed up, you can boot with a Linux CD, mount the partition containing /root (hda1 in this example) and put back the MBR with the command:

<file>
dd if=/mnt/hda1/root/hda.boot.mbr of=/dev/hda bs=512 count=1
</file>

===== Getting around file size limitations using split =====

When making images, it's possible to run up against the operating system (filesystem) file size limitations. One way to work around a given file size limitation is to use the split command.

<file>
dd if=/dev/hda1 | gzip -c | split -b 2000m - /mnt/hdc1/backup.img.gz
</file>

This example is using dd to take an image of the first partition on the first hard drive.

The results are passed through to gzip for compression . The -c option switch is used to output the result to stdout.

The compressed image is then piped to the split tool . The -b 2000m switch tells split how big to make the individual files. You can use k and m to tell switch kilobytes and megabytes (this option uses bytes by default).

The - option tells split to read from stdin. Otherwise, split would interpret the ''/mnt/hdc1''... as the file to be split.

The ''/mnt/hdc1''... is the prefix for the created files. Split will create files named backup.img.gz.aa, backup.img.gz.ab, etc.

To restore the multi-file backup, do the following:

<file>
cat /mnt/hdc1/backup.img.gz.* | gzip -dc | dd of=/dev/hda1
</file>

Cat recombines contents of the compressed and split image files to stdout, in order. Results are piped through gzip for decompression and are then written to the first partition of the hard drive with dd.

===== Image one Disk to Another over a Network =====

http://www.pantz.org/software/dd/drivecopywithsshanddd.html

First, determine which disk to work with and zero out the unused space on the source disk to make compressing the image much easier:

<file>
lshw -C disk

dd if=/dev/zero of=0bits bs=20M; rm 0bits
</file>

Then boot any bootable linux distro (like sysrescuecd or a CentOS Live CD) on the source machine (holding the disk you want to image) and do something like the following.

Create a compressed disk image as a file on a remote (perhaps intermediary) host:

<file>
dd if=/dev/sdx | gzip -c | ssh user@hostname 'dd of=image_of_sdx.img.gz'
</file>

Copy (image) a disk (securely) directly to a disk on a remote host:

<file>
dd if=/dev/sdx | gzip -c | ssh user@hostname 'gunzip -c | dd of=/dev/sdx'
</file>

Assuming ''sdx'' is the destination drive on the destination machine, the following restores the image to a local disk on the destination machine. To restore the image boot a Linux distro on the destination machine to restore and pull the image that you created and dump it back with the command:

<file>
ssh user@hostname dd if=filename.img.gz | gunzip -c | dd of=/dev/sdx
</file>

This will usually take a few hours so be prepared.