This should hopefully be, the final part of what has turned out to be a three part series documenting my thoughts on managing my laptop’s disk resources. It all started when I swapped my old laptop hard disk for a superb 500GB Seagate Momentus XT Solid State Hybrid Drive to help alleviate my disk space woes last year and then started experimenting with virtual hard disks (VHDs). Initially everything was fine but it’s taken me a while to fine-tune and accommodate a couple of new considerations posed by VHDs. This post formalises what I’ve (finally?) settled upon, so that I can manage my various laptop OS/working configurations to give me the best flexibility for the future.

Part one of this (unintentional) series, was about moving my working environment to a new disk with minimum of fuss [You can read that post here:How to turn a physical disk into a bootable VHD”]. This proved a highly successful experiment, enabling me to copy my existing laptop configuration to a VHD and then be able to dual boot into the rebuilt configuration (native partition – native OS) or run my old configuration through a contained VHD directly on the laptop hardware (no need for VMWare or Virtual PC). This was achieved with little downtime and also provided an unexpected backup strategy too. However, even though I had transferred and converted my old configuration into a single VHD file (see diagram below) it had not actually solved any long term space issues – the original partitions still remained practically the same size and were pretty much bursting (forcing me into strange backup procedures to ease the discomfort – see my post “Windows Phone 7 Backups – Release more space”].

 Blog post 1 got me to here . A bootable VHD residing on the physical disk.

Part two talked about how to extend or shrink a VHD using Windows 7 [You can read that post here:Resizing Virtual Hard Disks (VHDs)”].  By making good use of the various actions available in Windows 7 Disk Management, I managed to reconfigure my VHD layout — to a point where I got stuck at a dead end. In my eagerness I thought I could just reduce one partition and expand the other and forgot the most basic rule of partitioning…

“a partition needs to be one contiguous block,
regardless of the disk drive being physical or virtual”

Damn. I had remembered the hard way, that although I had made space by reducing the excess of my Data partition, the free space was in the wrong position to extend my nearly full OS partition (see diagram below).

 Blog post got as far as here. Wasted unallocated space on the bootable VHD.

So now I’ve had a bit of time to take stock of the situation, experiment a bit further and unravel the sequence of getting my partitions to the desired sizes. While this notion of cloning to VHD is a feasible option it does come with a couple of restrictions that may alter your course of action:

  1. Mounting a cloned VHD within the originating VHD is not possible – it’s a bit like a Star Trek time paradox. The cloned VHD will have the same logical disk identifier (which is embedded within the disk identification metadata) as the parent it came from. So even trying to mount one inside the other will cause a clash of IDs for which, Windows will prevent. This harks back to the time when physical disks ruled and were copied rather than cloned.

  2. VHDs are actually limited to 128GB in VMWare or Virtual PC. I did not expect this one, but it’s true. I’ve survived with a 155GB VHD only because I use it as a bootable VHD direct from a native Windows instance as explained in my previous post “How to turn a physical disk into a bootable VHD”.

  3. Running the cloned VHD within an instance of VMWare or Virtual PC with the originating drive running as host. Paradox time again, you’ll bump into the issue of logical disk identifiers again (as explained above) and doubly unlucky if over the 128GB limit (as explained above).

  4. VHDs are only guaranteed to be mounted and ready if you boot from them. Unfortunately, Windows does not remember what extra VHDs you attached and does not remount them after the Windows start-up sequence has completed. Impracticable - splitting to separate VHDs means one will not remount automatically on reboot.So, in the adjacent diagram, splitting my one large VHD file into two separate VHDs (so that I can expand or shrink the partitions independently) the first VHD file will always be available because it’s what I boot from. The second VHD will have to be remounted each time (hence the big red cross). You can script this, so that Windows remounts this automatically, but I moved my profile and other various files to the second partition, so scripting is not good enough since Windows might need files from this VHD before it is mounted.

  5. Reverse the procedure and go “VHD to disk” ??? – unfortunately this is not possible. Moving from either a fixed size or dynamic VHD back to a physical disk’s partition set is not possible. There are no native tools available (Disk2VHD does not go backwards!). There are a number of other backup tools that perform this action, but this is not actually a backup scenario in itself so I won’t count them as an option.

 

Disk management can be like a slider puzzle.In fact it feels like I’m playing one of those tile sliding puzzles, juggling disk resources about, trying to mount this VHD, that VHD, rebooting into a new native operating system and so on. So, having stepped back for a bit more thought I reverted back to first principles of disk management as if I was swapping physical drives rather than virtualised ones. So, to recap the problem, I need to reduce my cloned VHD to a more sensible 120GB (under the 128GB limit) and expand the C:\ logical partition upon that VHD. The following steps are how I overcame this bearing in mind the above restrictions….

 

Firstly, defrag, defrag, defrag! Remember i mentioned contiguous space for partitions? Well the same can be said within partitions; if there is any disk/partition resizing to be done, it’s best done with all the files pushed into one contiguous block. There are a number of disk tools that can do this but not all work well or at all with VHDs. So I would recommend using Auslogics Disk Defrag (free for home use) and running the tool with the “Defrag & Optimize” option which help reorder the files on disk.

 Auslogics Defrag & Optimise option

Once satisfied with the system being as optimally ordered as possible, the next step is to offload the “blocking” partition. The second partition was emptied over to another disk, preferably a local physical disk – not another VHD or network location because that would take too long. I can reply on a basic file copy for this partition because its only a data; there is no boot sector here. Also, when copying the files away from the source, forget about using Windows Explorer’s drag and drop since it seems a lot slower and only copies what you see (unless you have reconfigured the settings for hidden and system files). I would recommend using the command prompt’s XCOPY command with the options to copy all hidden/system files/folders as required. The larger the amount of data the longer it will take, so probably left best to cook overnight!

 Making room. Backup the second partition to local disk elsewhere!

Now the data partition is safely backed up, I want to delete my second partition so that it it will present the minimum amount of hassle with resizing my VHD in the next step. If I did not, I would have to reduce the last partition to a suitable size that allows the overall disk to shrink further. When shrinking a partition, its more complicated because the shrinking process is limited to working with unallocated disk – hence my suggestions to defrag and optimise the disk beforehand. Since my OS partition is well below my intended 100GB VHD target, removing the Data partition enables me to resize with impunity. Windows 7 comes with a decent set of inbuilt disk tools (as I mentioned in “Resizing Virtual Hard Disks (VHDs)”) to allow you to extend, shrink and delete partitions (Volumes). Just type “Disk Man” in the search box from the Windows Start menu and select “Create and format hard disk partitions” – the Disk Management Console will look a bit like the next diagram.

   The Windows Disk Management console.

Having deleted the second partition, shrinking the VHD will be much quicker. Reboot into a Windows OS instance that is not run from the VHD and ensure that the VHD is un-mounted – if not already. Run VHD Resizer (its’ an old tool with a very basic UI but it does the trick) selecting the VHD to shrink and the destination for the new VHD (it does not alter the original). You’ll note that when you select the source VHD, VHD Resizer will think for a while – basically it is analysing the source VHD to determine what is the maximum amount of shrinkage that can be allowed. Type the new size desired (100GB here) and ensure that the “Type” dropdown is “Fixed” because in this case my VHD is a Hyper-V/bootable VHD and they only work with fixed sized VHDs! If you have space, use a local destination (another partition, drive or USB hard disk) because the tool will want to create a 100GB file and that’s quite slow over a network. Be prepared, this process will take some time to complete.

Removing unwanted partitions makes VHD shrinking a lot easier.

The VHD should now be a neat 100GB and still with lots of free space because the second partition was previously deleted to allow a a contiguous extension. I need to bump the size of up my OS partition now, so by using the in built Windows 7 disk tools (see previously) it’s simply a case of right-clicking my partition and selecting “Extend Volume” (I won’t take you through the wizard steps because its really easy to figure out). I’m being selective about what I put back onto my second partition, so I’ve planned a 60GB/40GB partition split.

Expanding the primary partition (finally!).

It does not take long to expand a partition (given some free contiguous space), so once that is completed use the Disk Management tools to “Create Volume” in the remaining unallocated space. Windows will automatically format and assign a drive letter; if the drive letter is changed a reboot will be required. Since the two partitions reside on the same VHD, they will both be available if you boot from that VHD.

Now copy whatever data is required from the second partition backup back to the new partition. As before, use the command line XCOPY to transfer the files across since it’s faster. When this all finished, I swapped the new 100GB VHD for the old 155GB VHD being careful in my renaming. The new VHD is essentially a smaller clone, so the boot sequence is none the wiser that the VHD size has dropped.

Nearly there, restoring the data files back to the VHD. 


In Conclusion: Neat and tidy. Feels like I've done the Rubik's Cube 1000 times over.
Well gosh, this has been a road of rediscovery but I’m glad that I’ve done it. The fundamentals of disk management are not that much different between physical disks and VHDs but there are some gotchas to plan around. These have not curbed my enthusiasm for bootable VHDs since it’s still an excellent to maintain separate bootable environments from one PC without having to resort to dedicated dual boot allocations and locking down to that PC. I use my native OS for general everyday computing and a bootable VHD for whatever client project/environment I need to at that time. As the environment gets old or the project finishes, the VHD can be archived and a fresh one created in it’s place. Neat.

It’s not every day, month or even 6 months that I need to go through all this juggling – only because this VHD was a mega workhorse of an environment, namely my old laptop operating system configuration! Keeping more than one bootable VHD on my PC has taught me other tip though; keep your local data to a minimum. If you don’t look at a folder or file in over a month, then archive it and don’t make it part of the project VHD, keep it native to disk and let the VHDs dip into it. This is how I ended up with a smaller second partition on the VHD because my ratio flipped from 30/70 to 60/40 operating system to data.

I think that as I finish writing this blog post (of three) this is much less about following a set of steps to manage your system but more of a brain dump of the obstacles that I faced that you may not have appreciated till now. Believe me, hitting a dead end after waiting 14hrs or so for a series of partition operations to complete is not fun! Hopefully I have spared you (some of) that frustration.

 

This is a cross post from my EMC blog, mainly for backup duplicity and to aggregate some of my past postings. My EMC blog used to be under the Conchango brand but was acquired by EMC so I’ve also retrospectively refreshed some of the old links and maybe a tweak a bit of content too.
permalink to the original post here