Welcome to the Cisco Academy for Vision Impaired Linux Wiki. |
Main /
Eyes-FreeDiskAndPartitionCloningDisk Backup Cloning and Creating Images with GRMLNote: this guide is based on Kerry's lecture 17, and various experiments I've tried afterwards. It may contain errors, and isn't guaranteed to work. I too, am a novice at these things. I have, however made my best attempt to ensure these procedures work, and they certainly will give you a solid start. -- Deborah Armstrong, June, 2012 What is GRML and Why Use it?GRML is a Linux distribution that runs entirely from a CD, though it also can be booted and run from either a USB stick, external hard disk, or internal hard drive. It's a "live" distro, meaning that it doesn't have to first be installed and will be a fully functioning system after booting. GRML has a minimalist graphical environment but is primarily based on the command line. Its intended audience are intermediate and advanced users with an interest in textual tools and system administration. Because many of these tools run from the Linux character console, it's particularly handy for the blind techie. Additionally, GRML has the speakup screen reader built in to its kernel, and also contains Brltty, making it fully accesssible to users of speech and/or Braille. It supports software speech, so no external, hardware synthesizer is necessary -- though one can be used if desired. Sighted users will often perform similar steps using the popular live Linux distro Knoppix. Knoppix isn't accessible by default. What is Disk ImagingThis process makes a backup of your system that can be restored after the disk is formatted, or on another hard disk. Periodically backing up to an image gives you a way of easily rescuing a broken system without needing to re-install and configure everything again. Because disk cloning, another name for the same process, takes a snapshot of your drive, it captures the entire operating system, and all software installed and configured, along with all the data on the drive. Users can choose to clone the entire hard disk or simply image a single partition. Commercial Windows programs that image disks include DriveImage, Drive Backup Express, Norton Ghost, Acronis TrueImage, and the shareware program, Image For Windows. A full list of software for many operating systenms can be found at: http://en.wikipedia.org/wiki/Comparison_of_disk_cloning_software and http://en.wikipedia.org/wiki/Comparison_of_disc_image_software and http://www.ehow.com/list_7242923_list-disk-imaging-software.html The drawback with most of these solutions except for Image For Windows is that they aren't very accessible. And when you boot the rescue CD to restore a broken system, it will not be running a screen reader, and thus will not be eyes-free. Though cloning with GRML requires a bit more knowledge, and a comfort level with the Linux command line, it gives you the freedom to know that you can restore your system without needing a sightling to assist you. PreparationIt goes without saying that anything you can do beforehand to make the image smaller is useful. Remove large files like pictures, MP3s, movies and other downloads if possible. You want probably to keep a snapshot of your running operating system, and not your entire iTunes media directory. ! I typically move all the data that I possibly can to an external hard drive and additionally burn it to DVD. This way my videos, music, Outlook folders and various things I've downloaded are off the hard disk but still safely backed up twice. It's also a good idea to defragment your drive, especially if you have tools that move data to the beginning of the partition. There are also free tools that will simply locate unused portions of the disk and fill those with ASCII zero. Normally random bytes reside in these locations. But converting every byte to ASCII zero gives contiguous strings of data that are easier for compression utilities like BZIP2 and GZIP to reduce in size. Several Windows utilities will "zero out" unused space. The free eraser http://www.x-ways.net/winhex/index-m.html are designed for security needs; deleting all traces of files that were previously erased by filling all unused space with either random bytes or zeroes. Another technique which I have not tested with an NTFS drive, and may therefore only work in Linux is this series of commands: mkdir /mnt/mymountpoint Of course, you'll need to substitute your partition name in the example above. In general, hard disks have the remaining traces of bygone files stored in sectors now allocated by the operating system as unused. Those bytes aren't removed until the file system assigns new data to populate that area of the disk. For Windows using NTFS, I personally prefer Sdelete, a utility with which I'm most familiar. Obtain it from: http://technet.microsoft.com/en-us/sysinternals/bb897443 Then type sdelete -z c: to convert all free space on that logical drive to bytes of ASCII zero. Another utility recommended by Cavi students is CCleaner at: http://www.piriform.com/CCLEANER this is a complete Windows optimization utility, and zeroing out unused space is one of the options you can check in its advanced tree view of configuration preferences. Besides preparing your system, you'll need a place to store the image. You might use an empty partition, or an external hard drive. You can temporarily install a second drive in your PC as well, or connect the external drive via USB or Firewire. Of course you can also use PATA or SATA cabling to your bus. Advanced users can read up on how to simply use another computer on their network as a target, either through ftp, samba, ntfs or ssh through netcat. There are several online guides that will walk you through setting up a remote machine to receive an image being compressed on the fly. If you are simply cloning from Partition X to Partition Y, then you don't need to format the partition. If you ar saving the image to a named file, then it needs to be on a partition formatted with a file system. Getting GRMLThe main website is at http://GRML.org/ From there you can link to a particular mirror in your own country for the fastest download. GRML has several flavors, but only one, the "full" version contains the speakup modules, so do not download the smaller flavor. You can also download an older version from the cucat website. Older versions are also available at various GRML mirror sites. As of this writing, the most current version is 2012.05. The previous version was 2011.12. Both of those use the same method for loading screen access. The newer versions are suitable for newer hardware. If you have an older PC, you might wish to try an older version of GRML. Older versions were actually larger; drivers for obsolete hardware and applications that were considered depricated were recently removed from GRML to streamline the distribution and make it more relevant for system administrators. However, older versions of Linux in general use less memory, and can often work better on hardware with limited RAM. An example of an older version which works well on older hardware, and uses the alternate method described below for loading speakup is GRML-2010.04. Note that versions are based on the release date. For purposes of this guide, the term "older" refers to versions 2011.5 and below. Burning The ISOGRML, like most Linux distros is an iso file, an image itself which must be burned to a CD using a tool that understands the ISO format. The built-in CD burning wizards for Windows XP cannot burn an ISO, but the free ISO recorder, at http://isorecorder.alexfeinman.com/isorecorder.htm can. ISO recorder is accessible and easy to use. Simply right click on the iso, and select write image to CD. For the versions of Windows 7 that are not limited, you can right-click on an ISO and select "burn ISO to CD". You won't be able to do this with other versions of the Windows operating system and will therefore, always require third-party utilities to burn ISO images. Other CD writing software, like Nti, Roxio and Nero can also burn ISOS but you must be running full and not trial versions of these programs and be willing to work around their accessibility quirks. If you are burning an ISO from a Linux system, the process is easiest of all; simply google for command-line tutorials for your specific distribution. Commonly used programs include cdrecord, mkisofs, genisoimage and wodim. Booting GRMLOnce the ISO is burned, you have a bootable "live" Linux distribution. Your PC may or might not boot from CD; if it cannot, you will need a sighted person one time to change the boot order in its BIOS. After that, the PC will default to booting from a CD first, so if the hard disk is dammaged, you can always boot GRML without assistance to restore. And if you wish to boot to the hard disk as you normally do, simply ensure that no CD is in your drive when you boot the system. It is quite possible that you could boot from USB, and there is a script in the GRML distribution called GRML2USB that formats a USB stick and sets it up with GRML to automatically boot with customized settings. The problem with this approach is that it is hard to know what's happening because a USB stick boots silently. It is reassuring to hear the swish-swish of the CD spinning; to know when you are at a boot prompt, or a menu, or when the system finishes loading. When GRML loads, you are presented first with a boot prompt, at which you enter "cheat codes" -- command line parameters that are processed by the system as it boots. You can simply press ENTER to bypass this prompt or do nothing, and GRML will load automatically with its defaults. GRML helpfully beeps the PC speaker at a boot prompt, so it's fairly easy to figure out what to do. However, at a boot prompt, you only have about 30 seconds to start typing, or it will time out and simply finish loading GRML. At the boot prompt the word grml is already filled in for you. Very old versions of gRML from before 2010 required you type grml, or some other boot target. The grml CD has several other boot targets which are not relevant here. So in most cases, at a boot prompt, you simply press tab, which acknowledges that you are executing the default boot target which is labeled grml. Then you are able to type boot parameters. For an older version, that parameter is swspeak and for newer versions, you can simply skip the tab thing and just press ENTER or wait for it to automatically finish loading. For any version, to load Brltty, you press tab to get to the edit field for typing boot parameters; then you type Other useful parameters include you supply the password as in A full list of cheat codes is at: After the system loads -- you'll hear the CD spinning, then stopping a few minutes later -- a rising tone will signal, in the newer versions of GRML, that a menu has appeared. Older versions may speak a message. If there's a menu, press Q and the ENTER key to exit it. You are then at a shell command prompt. For an older version, you are already at a shell prompt, though there is a screen of helpful information already displayed. For these older versions, type yes, that's the same thing you already typed, but this time you are asking GRML to run a shell script with the same name. The swspeak cheat code told it to load a specific kernel module; this second swspeak calls a shell script. If you are successfull, speakup will then be fully operational, and the system will start talking. For newer versions of GRML, type This loads the speakup modules for software speech in to memory. Press ENTER. Then type This loads software called a "connector" which interfaces the speakup screen reader with the espeak software speech synthesizer. You are now running a talking command-line, text-based, sometimes called console Linux. If the volume isn't high enough, for all versions, you can try these three commands: amixer set Master 100%,100%,unmute If the device whose volume you are attempting to raise doesn't exist, you'll get an error, but do try all three of these commands before giving up! GRML is now running from the CD but has copied substantial portions of the operating system in to RAM. A few tips on using SpeakupA few basic keystrokes are good to know: To read by lines, the numeric 7, 8 and 9 keys will read the previous, current and following line respectively. The numeric 4, 5 and 6 read by words, and the 1, 2 and 3 by characters. The most useful speakup keystroke I find is caps lock I which reliably reads the current line whether or not you're using a laptop. On my laptop, because the FN key held down with I gives me the numeric 5 which reads words, I often simply press fn-I twice rapidly to phonetically spell the current word, then hit capslock I to read the current line. To increment the punctuation level (until is reads all punctuation) press the F10 key while holding insert or caps lock. To increment the "reading" punctuation level in the same way hold insert or caps lock while repeatedly pressing F12. Speakup has two different punctuation settings, the "reading" and the regular punctuation and defaults to setting them both at level 2. Four is the highest and best when doing critical work, like imaging. Holding Insert or caps lock and pressing F5 and F6 decrement and increment the speech rate respectively. Lastly, holding insert or caps lock, and pressing F1 gives you brief help; a list of keystrokes you can arrow through. One way to practice reviewing the screen with speakup is to load up a man page and get comfortable navigating with that first before trying to review crucial commands. Booting summaryOlder Version: At the beep, press TAB, type swspeak and press Enter. Wait. After it stops spinning, type swspeak once more, and press enter. Newer version: At the beep press Enter. At the three rising tones, press q and Enter. Type Getting the Disk Device NamesYour next step is to figure out what device names were given to your drives. For example, your first drive might be /dev/sda and the second drive /dev/sdb. But you want to be absolutely sure, because you want to copy the correct drive to the right destination. Commands that can help you are fdisk -l which lists all your drives and partitions. For example /dev/sdb3 is the third partition on the second drive and /dev/sdc2 is the second partition on the third drive. Note that Linux has its own way of counting first second and third. It might not be your way; an external, portable USB hard drive might be the last drive, as in /dev/sdf, but you cannot be sure. For example, /dev/sda1 might be the CD drive, with the GRML live system. And I've seen USB drives that boot up as /dev/sdb4. Peruse the output of fdisk -l which shows information about each drive, such as its size and number of partitions to help you figure out which drive is the source, and which the desired destination. This command also gives you valuable information about your SCSI drives. You can also mount a partition and take a look at the files there to ensure it's the correct source. For example: The third example uses another Linux program called mount.ntfs-3g(that's from the ntfs-3g package) which does a more reliable job than the kernel alone for reading and writing ntfs volumes. Basically, if you are just looking at data, it's find to use the standard Linux mount command. But if you plan to change NTFS data, such as when creating an image on an NTFS-formatted external hard drive, then it is important to use the mount.ntfs-3g variant. The third above example mounts a Windows partition. Note that the /mnt/sdb1 is simply a filename. You could have type mount.ntfs-3g /dev/sda1 /windowsdrives/firstone provided the directory /windowsdrives/firstone existed and that /dev/sda1 was actually a windows partition. GRML already has set up a structure in /mnt for you to mount your drives. Newer GRML versions often find your Windows and Linux drives already and automagically mount them for you. In this case, it will set up a series of symlinks to your devices, giving them names based on their type and size. You can use the file command to get more information about one of these, for example: file /mnt/500G Typing simply Note that a drive and/or partition can exist without being mounted. Mounting simply is the Unix term for getting it attached to the file system. Also note that /mnt is simply a convention for where drives are often mounted. GRML uses this convention, but you could just as easily mount a drive anywhere you choose, for example creating a directory called MYUSB and mounting the USB drive there. Now go ahead, and change to the /mnt/sda1 directory and do an ls to look at the directories there. Of course if your drive is mounted elsewhere, substitute the appropriate directory. Continue to poke around, examining all drives and partitions that GRML located on your system. Ensure you know which device is the one you wish to clone and which is the destination, where the image will be saved. remember the names are vollitile. Once you've figured out what goes where, ensure that you are on a mounted drive and your default directory is writeable and has enough space to hold your image. Often this will be an external drive. The Note that the percentage displayed is the used space. For example if 82% is displayed you know that 18% of your drive is free space. Here are a few more examples: For examining space, the du command shows the disk usage and df the free space. The -h parameter gives this to you in "human-readable" format, as in megabytes and gigabytes. If you find that you need to format a partition that you'll use as a target, the easiest command to work with is parted. You'll want to unmount the partition first and read up on parted a bit to learn its commands. The ntfs-3g package also can format ntfs drives and the old mkfs commands (make file system) are still available to create files systems for etxt2 ext3, reiserfs, hpfs, ms-dos, and fat32. Commands to Image a driveThe oldest and most universal command to accomplish this is dd, which stands for "data dump". Its syntax is The -bs specifies the block size, which for our purposes, we will keep at 1. the -if specifies the source, what you are copying from. the -of specifies the target or destination. After the -if and -of is an equals sign, followed by the device or filename. It's important to not confuse the input with the output, and for good reason, dd has been jokingly called "disk destroyer". The source should not be mounted, so you'll want to use umount first, but the target must be mounted. For example: If your target is a partition, rather than a file, you won't want to compress it of course, but you'll want to ensure that it's either the same size or larger than the source. If your target is a file, compressing it makes sense. BZip2 is a unix utility that's universally available and makes tightly compressed images. You could also use tar, but it handles sparse files less efficiently, or gzip, which runs faster, but compresses less tightly. A more modern utility for cloning Windows file systems , is appropriately named ntfsclone. A sample command line might read: ntfsclone -O - -s /dev/sda1|bzip -1 >ntfs.gz In this case, gzip was used to compress the image. NTFSCLONE has clear documentation both online and on GRML. Unlike DD, which simply copies blocks of data and is file system independent, NTFSClone is able to ignore the unused portions of the disk. But ntfsclone works only with ntfs file systems. Other imaging and helper utilities on GRML include dd_rescue and the similarly named ddrescue. There is also dcfldd, mondo rescue, clonezilla, partimage, ntfsresize, and Partclone. Before using any cloning software, you should study its documentation and ensure it's right for your needs. For example, though NtfsClone will image a Windows partition, it won't resolve any booting issues. If you are expecting to move that image to a drive in a different location or of a different size, you will need to fix the partition table so that the correct partition is flagged as bootable, and depending on the boot manager you are using, it also will need to be configured to correctly boot your chosen partition. The command-driven parted utility can be helpful here. Also check out ntfsresize, especially if you need to move a larger system to a smaller hard disk. A fresh install of Windows 7 creates a 100MB boot partition. So your drive C is not, therefore the first partition on the drive. To save the partition table and the master boot record, type That'll get the first 512 bytes, giving you a binary image of the primary partitions' info and the master boot record. You might want to save more than one sector, because your system uses a secondary boot loader. For this situation, you'll use a command like: To back up the extended partition information, type: Linux often uses extended partitions; by default Ubuntu and Debian both do. But Windows typically does not, though many OEM hibernation, backup and restore utilities do. If your PC has a hidden partition with the manufacturer's system files, it often is an extended partition. You may or may not need to keep this data. Simple Restore Examples Asuming you have your external drive attached, and know which partition or drive you wish to restore to, and that your image is compressed with bzip2, these examples could work: bzcat sda_sectors.bz2|dd bs=1K of=/dev/sda bzcat image_sda1.bz2|ntfsclone -r -O /dev/sda - This example: To restore your partition table and master boot record, do: If you only want to restore the actual MBR code and not the entire primary partition table, just restore the first 446 bytes: To restore the extended partition information, type After you restore boot information and partition tables, it's a good idea to simply run fdisk, and to exit out of it to force it to reread the new partition table. Do this before restoring the actual file
fdisk The reason you do this is so that the running GRML knows how the new partition table is configured. The fdisk step isn't necessary if you are restoring the boot info and partition table along with the entire file system to make an exact copy, but it is necessary if you used ntfsclone. One restore concern I had was how to cope if I wasn't sure how big my partition table and boot information was. For Windows, I discovered I could simply restore the boot information first, then restore my entire partition afterwards. Thois isn't going to work if you dual-boot but for most backup sitsituations using Windows, it's acceptable. ReferencesHere are some advanced command-line examples and a list of programs that do backup, imaging and copying to and from servers on a network: Here is a basic tutorial on using dd: How to set up disk cloning with dd, netcat and ssh tunnel http://serverfault.com/questions/51567/how-to-set-up-disk-cloning-with-dd-netcat-and-ssh-tunnel Using DD for Disk Cloning -- a good forum discussion: http://serverfault.com/questions/4906/using-dd-for-disk-cloning Notes on Backing Up Hard Disks or Entire Partitions-- a blog post: http://www.inference.phy.cam.ac.uk/saw27/notes/backup-hard-disk-partitions.html Backing Up Your Hard Drive Over The Network With SSH -- a blog post: http://patrick.wagstrom.net/weblog/2009/05/08/simple-full-disk-image-backups-with-dd/ Disk Cloning Basics -- a forum discussion thread on dealing with a dammaged file system http://ask.metafilter.com/205058/Disk-cloning-basics-using-dd-in-Linux Using DD Over Netcat with SSh -- a set of benchmarking tests http://www.ndchost.com/wiki/server-administration/netcat-over-ssh Copying Windows To a New Drive Using Linux -- A Tutorial http://www.nilbus.com/linux/disk-copy.php Ghost Your Windows System Using Open-source Tools http://tipsfor.us/2007/06/05/ghost-your-windows-system-for-free-using-open-source-tools/ Using NTFSClone -- a tutorial Transfer Windows To New Hard Disk Using NtfsClone -- another good turtorial http://edoceo.com/exemplar/ntfsclone-transfer-windows Mount NTFS Drives With Read-Write Access -- an explanation of the NTFS-3g tools http://linuxconfig.org/How_to_mount_partition_with_ntfs_file_system_and_read_write_access Mounting NTFS -- documentation that's short but especially clear Note that the above references often describe limitations and bugs in ntfsclone that have, since the writing of these tutorials have been addressed. ConclusionThe above are simply examples, and not tailored for your particular partitions, drives' device names or file systems you wish to clone. Do experiment and research first and don't cut and paste any command from any guide. It is also wise to read tutorials online, including this one critically, because people do make mistakes and information is often dated or simply inaccurate. Deborah Armstrong, June 2012. |