Corrupted bitlockered VHD!

How can a VHD become corrupted since it's just software?
Can this be fixed at all?

Peter Jason wrote:
How can a VHD become corrupted since it's just software?
Can this be fixed at all?

And your backup copy good sir, after I warned you ?

*******

VHDs come in at least two flavors.

They can be fixed in size.

They can be dynamic, and they grow as they fill.

They're kinda sparse in the sense that the storage
quanta is something like 2MB, and storage would be handled
in "chunks".

Any section of a VHD that remains as "all zeros block" doesn't
need a representation in the VHD. That's how it would
be *unencrypted*.

To be encrypted, the whole thing would be randomized, including
the zeros blocks. Thus a 40GB partition takes 40GB right away
and might as well be fixed. There'd be no point in making
a dynamic one, because after the encryption step, a 40GB virtual
volume would take precisely 40GB of space. Even if the partition was empty,
after encryption it's still perfectly random looking 40GB of data.
There should be no hint that an encrypted volume is empty.
It looks like random data no matter how orderly the data
inside it actually is.

Does Bitlocker have a scheme to encrypt various parts of the
file system separately ? Would it even make a difference ?

And if I had to guess what this corruption was, it could
be related to Microsofts fetish with not keeping $BITMAP
updated properly, and not having $MFT and $MFTMIRR maintained
as they would have been in WinXP.

Then the question becomes (and this is important):

Where (under what circumstances) was this corruption detected ?

"The program with the problem" is important!

It could be that some program has the usual problems
with the integrity of the Windows 10 version of NTFS.
Note that Macrium has had a few problems, as an example.

If it's corrupted in File Explorer, then I can't see this
ending well. An error in any metadata structure, even a
one bit error, could cause a relatively large randomization
of surrounding data. These structures would all need to be
duplicated and stored somewhere so the backup copy would
not be hit by the primary metadata failure.

You also have to ask yourself, how could such an error be
introduced ?

We rely on storage devices to meet their ten to the minus fourteen
probability of errors being present. And the encryption scheme
needs some way to tolerate such problems. I don't know if
they include parity blocks or any other similar scheme to
cover for the impact of encryption plus the usual noise
floor on storage.

Bitlocker on Windows 10, doesn't have diffusion, so a bit
error should have a lesser impact than it might have
had on Windows 7. But that's not going to make you feel
any better. The massive mess flipping a bit in a Bitlocker
volume *should* make, I can't see how CHKDSK can fix it.
It would be like a train wreck in there, if the bit error
is in the head end of the file system (near where all the
metadata parts were stored).

*******

A suggestion here is "repair-bde".

https://social.technet.microsoft.com...7itprosecurity

https://docs.microsoft.com/en-us/win...nage-bitlocker

"Repair-bde can reconstruct critical parts of the drive and
salvage recoverable data as long as a valid recovery password
or recovery key is used to decrypt the data. If the BitLocker
metadata data on the drive has become corrupt, you must be able
to supply a backup key package in addition to the
recovery password or recovery key."

Based on "naming style", that's a Powershell executable, so you'd
want to run Powershell before launching it.

I stored my recovery info for my test bitlocker, in a text file
for easy retrieval.

This is an example from the Vista era.

https://support.microsoft.com/en-us/...a-from-an-encr

# Damaged (RAW) D: recovered to being stored in E:
#
# Obviously E: should be some other hard drive, as you want to
# follow the "no writing to the damaged disk" rule.

repair-bde D: E: -RecoveryPassword 1234567890123456 === some numerical password
from the recovery file ?

And to think I removed my encrypted volume only yesterday,
because Gparted wouldn't move it for me :-/ I hope I decommissioned
my test properly and didn't mess things up.

Anyway, that's something to get you started. I'll have to
cobble together another test, and see how well this
repair thing works.

Paul

Peter Jason wrote:
How can a VHD become corrupted since it's just software?
Can this be fixed at all?

So, to simulate this, I selected a block of 512 bytes on
my VHD and zeroed them.

Of course, the next time I tried to "attach" that VHD,
it claimed it was corrupted.

Now, it was time to use repair-bde. And guess what ?
It needs a drive letter, like H: , to repair and decrypt
to take the place of M: .

repair-bde H: M: ...

OK, but the VHD won't mount, because H: has been damaged by my
modification to the VHD file. I can no longer "attach" it in
Disk Management.

I fired up Virtualbox, and made the VHD for M: one of the
storage devices in VirtualBox. I booted a Ubuntu LiveDVD
in the virtual machine I was setting up. I used "dd" to
grab the second partition.

VirtualBox storage devices

CD/DVD drive = ubuntu.iso
/dev/sda = broken.vhd
/dev/sdb = transfer.vhd

Once in Ubuntu Guest, I transferred the sda2 partition
(the corrupted one). I can do this, because unlike grumpy
ole Windows, Ubuntu doesn't care about the partition
contents, as long as you don't click anything.

sudo dd if=/dev/sda2 of=/media/mount/transfer/transfer.img bs=1M

Once that finished, I shut down Ubuntu Guest in Virtualbox.

Then, "attached" transfer.vhd, which gives me access to transfer.img.

On a USB stick, I used Disk Management to make a single partition
which is the same size as the transferred image. 10433MB or close
to 10.5GB or so. The USB stick is 32GB and I'm using about a third of it.

Using the Windows port of dd...

http://www.chrysocome.net/downloads/dd-0.6beta3.zip

I then transferred the partition. This overwrites the partition
on the USB stick, without touching the MBR. 10.5GB of writes
at 90MB/sec.

dd if=M:\transfer.img of=\\?\Device\Harddisk2\Partition1 bs=1M

The benefit this brings, is before the partition was overwritten,
the USB stick was assigned a drive letter. It was drive D: .

I ejected the USB stick. Which turned out to be much harder than
I expected. I had to go into Disk Management and eject it from
there, as safely remove refused to work.

I plugged the USB stick back in, and the same letter D: was
assigned to the (damaged) volume.

I had a separate VHD file attached and it had a drive letter
of M: on it, and this was going to be my recovery volume. You
could use a hard drive for this if you wanted. I was just
farting around, so I didn't care that M: was another VHD file
container :-)

I used this command. It looks like the thing is just a
program, so runs in an Administrator Command Prompt.

repair-bde D: M: -rp 111111-222222-333333-444444-555555-666666-777777-888888 -f

The long string of numbers, was in my text file saved for
recovery purposes. When you make a Bitlocker volume, they
recommend some sort of recovery information.

After grinding for a bit, it says:

Finished decryption.

ACTION REQUIRED: Run ' chkdsk M: /f ' before viewing decrypted data.

CHKDSK found ten orphan files. So the volume that is
recovered, isn't exactly as it was before.

I have to figure out a way now, to detect differences
between the original volume and the recovered volume.
I'm considering "Hashdeep64" if I can figure out the
mess of arguments required.

Since your damage is likely logical versus the structural
damage I did on purpose, chances are your recovery will be cleaner
than mine. CHKDSK may whine a bit, but maybe you won't
have orphans.

Summary: The VHD idea sucks! You need to transfer the
partition that won't mount, onto a regular hard
drive or to a USB stick, and do it so Windows can't
see what you're doing. Windows refuses to attach a VHD
without mounting it in the same step. If it fails,
the "attach" attempt is rejected. Using a Ubuntu VM,
I could work on the VHD at my leisure and make
a copy of the sectors, to put on my USB stick for
recovery with the command above. You must be adept
at using "dd", adept at making partitions precisely
the same size and so on.

https://i.postimg.cc/jS8jqHxB/lots-of-volumes-used.gif

Recovery is possible, but not easy. Someone was asleep at
the wheel at Microsoft, when they "casually mentioned"
that you could BitLocker a VHD. Big deal! They probably
haven't added any code to repair-bde since 2013, and it
needed to have a VHD recovery option added. I don't
see such an option in the help.

Paul

Peter Jason wrote:
How can a VHD become corrupted since it's just software?
Can this be fixed at all?

It's frustrating, but I still can't get any tool to attach
a damaged bitlocker VHD and just leave it alone. So far,
transferring the partition using a Ubuntu VM is the best
I was able to do.

I tried to use OSFMount on the .vhd file. It pretends
it has mounted it, but it doesn't show up in Disk Management,
which means it did execute the "Attach" step, and the
OS returned an error to it "No can do". It eats the
error internally and in the OSFMount pane, shows
the VHD as being mounted. When it isn't.

I tried to use the "image transfer" inside OSFMount,
to transfer the buggered D: to a transfer.img type
file, and it got part-way then errored out.

That's as close as I could get to a "Windows-only"
solution. fail whale.

The recovery process will be a nuisance. It's not
hard to do, just a bit annoying is all.

Paul

On Mon, 21 Oct 2019 11:53:11 -0400, Paul
wrote:

Peter Jason wrote:
How can a VHD become corrupted since it's just software?
Can this be fixed at all?

It's frustrating, but I still can't get any tool to attach
a damaged bitlocker VHD and just leave it alone. So far,
transferring the partition using a Ubuntu VM is the best
I was able to do.

I tried to use OSFMount on the .vhd file. It pretends
it has mounted it, but it doesn't show up in Disk Management,
which means it did execute the "Attach" step, and the
OS returned an error to it "No can do". It eats the
error internally and in the OSFMount pane, shows
the VHD as being mounted. When it isn't.

I tried to use the "image transfer" inside OSFMount,
to transfer the buggered D: to a transfer.img type
file, and it got part-way then errored out.

That's as close as I could get to a "Windows-only"
solution. fail whale.

The recovery process will be a nuisance. It's not
hard to do, just a bit annoying is all.

Paul

Thanks Paul, but I'll just scrap it because I made it a week ago and
it has only a few files. Next time I'll ''age'' any new one to test
for stability.

Peter Jason wrote:
On Mon, 21 Oct 2019 11:53:11 -0400, Paul
wrote:

Peter Jason wrote:
How can a VHD become corrupted since it's just software?
Can this be fixed at all?
It's frustrating, but I still can't get any tool to attach
a damaged bitlocker VHD and just leave it alone. So far,
transferring the partition using a Ubuntu VM is the best
I was able to do.

I tried to use OSFMount on the .vhd file. It pretends
it has mounted it, but it doesn't show up in Disk Management,
which means it did execute the "Attach" step, and the
OS returned an error to it "No can do". It eats the
error internally and in the OSFMount pane, shows
the VHD as being mounted. When it isn't.

I tried to use the "image transfer" inside OSFMount,
to transfer the buggered D: to a transfer.img type
file, and it got part-way then errored out.

That's as close as I could get to a "Windows-only"
solution. fail whale.

The recovery process will be a nuisance. It's not
hard to do, just a bit annoying is all.

Paul

Thanks Paul, but I'll just scrap it because I made it a week ago and
it has only a few files. Next time I'll ''age'' any new one to test
for stability.

OK, right-click the .vhd file and open with 7-ZIP (from the
right-click context menu).

0.ntfs
1.ntfs size=10GB
2.ntfs

Each item is the byte content of a partition.

But my 1.ntfs (the encrypted one I wanted to recover),
fails with "Incorrect function", part way through the
transfer.

That would be one-half of the problem. Then you have
to stage the bytes of data on a real storage device.
That's where I used my USB stick.

+-----+-----------------------------------+------------------+
| MBR | 1.ntfs 10GB worth of bytes D: | |
+-----+-----------------------------------+------------------+

You create a new partition on the USB stick (in my case 10433 MB)
as the first step.

Then the contents of 1.ntfs are used to replace the contents of
the first partition on the USB stick. (You still use dd.exe
from http://www.chrysocome.net/downloads/dd-0.6beta3.zip for this.)

After that, the "repair-bde" program can be used to
extract and rescue the contents of D: .

Paul

On Tue, 22 Oct 2019 08:47:55 +1100, Peter Jason wrote:

Thanks Paul, but I'll just scrap it because I made it a week ago and
it has only a few files. Next time I'll ''age'' any new one to test
for stability.

I`m running VMs from encrypted VHDs for years without loosing any data
(Win 7, Win 10 as hosts). But my hardware is professional (Dell
workstations), and these VMs are always running from RAID1. A few days
ago I had the problem that one VHD did not want to remount after
backup, I had to restart the host (Win 10). But such problems are
rare.

Regards

M.

Paul wrote:
Peter Jason wrote:
On Mon, 21 Oct 2019 11:53:11 -0400, Paul
wrote:

Peter Jason wrote:
How can a VHD become corrupted since it's just software?
Can this be fixed at all?
It's frustrating, but I still can't get any tool to attach
a damaged bitlocker VHD and just leave it alone. So far,
transferring the partition using a Ubuntu VM is the best
I was able to do.

I tried to use OSFMount on the .vhd file. It pretends
it has mounted it, but it doesn't show up in Disk Management,
which means it did execute the "Attach" step, and the
OS returned an error to it "No can do". It eats the
error internally and in the OSFMount pane, shows
the VHD as being mounted. When it isn't.

I tried to use the "image transfer" inside OSFMount,
to transfer the buggered D: to a transfer.img type
file, and it got part-way then errored out.

That's as close as I could get to a "Windows-only"
solution. fail whale.

The recovery process will be a nuisance. It's not
hard to do, just a bit annoying is all.

Paul

Thanks Paul, but I'll just scrap it because I made it a week ago and
it has only a few files. Next time I'll ''age'' any new one to test
for stability.

OK, right-click the .vhd file and open with 7-ZIP (from the
right-click context menu).

0.ntfs
1.ntfs size=10GB
2.ntfs

Each item is the byte content of a partition.

But my 1.ntfs (the encrypted one I wanted to recover),
fails with "Incorrect function", part way through the
transfer.

That would be one-half of the problem. Then you have
to stage the bytes of data on a real storage device.
That's where I used my USB stick.

+-----+-----------------------------------+------------------+
| MBR | 1.ntfs 10GB worth of bytes D: | |
+-----+-----------------------------------+------------------+

You create a new partition on the USB stick (in my case 10433 MB)
as the first step.

Then the contents of 1.ntfs are used to replace the contents of
the first partition on the USB stick. (You still use dd.exe
from http://www.chrysocome.net/downloads/dd-0.6beta3.zip for this.)

After that, the "repair-bde" program can be used to
extract and rescue the contents of D: .

Paul

OK, I'm getting closer to a "Windows-only" recipe :-)
But more I/O is required than with my other method involving Linux.

1) Install VirtualBox. You won't be using VirtualBox so you
don't have to worry about that part. We're only after
a "utility" in the VirtualBox folder.

https://www.virtualbox.org/wiki/Downloads

# "Windows Hosts"
https://download.virtualbox.org/virt...133895-Win.exe

2) Now, look for "VboxManage.exe" in Program Files.

cd /d C:\Program Files\Oracle\VirtualBox

VBoxManage clonemedium C:\disk.vhd C:\disk.img --format raw

My original .vhd was maybe 7GB, but the "size" of the
virtual drive was 64GB total, so "disk.img" will need
64GB of storage space. It expands the damn thing.

3) Shut down. Connect a scratch hard drive to hold the
64GB image as a physical drive (it will hold a "converted"
disk.img).

Connect a second scratch hard drive (or even an unencrypted .vhd),
to hold recovered files. The drive will be re-written with
an image of the recovered materials, so you need a "clean"
drive with absolutely nothing you want to keep.

In fact, both storage devices should be "history free" in
that sense, as they're both about to be "zapped".

4) Now, it's time for "dd".

http://www.chrysocome.net/downloads/dd-0.6beta3.zip

Unpack it, to get at the dd.exe file

In an Administrator command prompt.

a) dd.exe --list

NT Block Device Objects
\\?\Device\Floppy0
\\?\Device\Harddisk0\Partition0 === whole drive pointer (starts at MBR)
size is 500107862016 bytes === size of that drive
\\?\Device\Harddisk1\Partition0 --- my copy of the VHD, a "broken" bitlocker
size is 1000204886016 bytes
\\?\Device\Harddisk2\Partition0 --- my "fixed" bitlocker contents end up here
size is 2000398934016 bytes This disk has the M: partition, covering the disk.

Make note of which drive is which. This is important!
The dd.exe program can overwrite *anything it wants* !!!

b) In Disk Management, Harddisk1 is the second row in the table.
I'll use harddisk1 to hold the converted disk.img for me.
Harddisk1 is the disk I will be presenting to Windows as
the "broken Bitlocker needing repair", and I hope to get
a drive letter from this process. Now, to transfer it in
1 megabyte chunks, as I propose here, I need to get the
precise byte size of disk.img and check that it is
divisible by 1048576 bytes. The bs parameter is the blocksize
of the transfer.

dd.exe if=C:\disk.img of=\\?\Device\Harddisk1\Partition0 bs=1048576

5) Now, in Disk Management, we want a Refresh from the menu,
so that Harddisk1 will be scanned and a new drive letter
will be showing up for the inaccessible ("unknown") partition.
Maybe the letter happens to be D: or something.

The subtle difference here, is how Bitlocker handles physical disks
versus the VHD file container. With the physical disk it seems to be
able to show the "broken" partition, without refusing to show the
disk drive in Disk Management. That's what we're relying on
in this procedure, is that a drive letter is assigned to the
broken partition.

6) With drive letter in hand for the one that won't open, now we repair
and recover into a recovery partition.

Let's say that M: is a suitably big partition on Harddisk2 (our
disk holding the recovered data). The M: partition will be
replaced, wholesale, by the repair operation pointed at M: .
This allows a (still corrupted from a CHKDSK point of view)
partition D: to be loaded in place of M: .

repair-bde D: M: -rp 111111-222222-333333-444444-555555-666666-777777-888888 -f

7) If step (6) warns you that CHKDSK of m: is required, you
follow the instructions it gives you

chkdsk M: /f

8) After that finished, your files are on M:

Examine the Command Prompt window output from (7) at this point,
for the details of "what files showed up as damaged" when
CHKDSK ran, to get some idea what files were lost.

HTH,
Paul