File Manager Copy/Paster Anomaly

This seems crazy to me.
I am trying to copy-paste a large folder (291GB) from C drive to G
drive (external). I have done this before with no problems. When I
do a 'Properties' check' of the folder on C-drive folder it says
291GB, When I do the same of the folder on G-drive it says 17GB.
There is room for 291GB on G-drive. When I try a copy-paste it
pre-counts past 291GB to 450GB, and says there is no room! And there
isn't. What cud be causing this?
Thanks
JW

wrote:
This seems crazy to me.
I am trying to copy-paste a large folder (291GB) from C drive to G
drive (external). I have done this before with no problems. When I
do a 'Properties' check' of the folder on C-drive folder it says
291GB, When I do the same of the folder on G-drive it says 17GB.
There is room for 291GB on G-drive. When I try a copy-paste it
pre-counts past 291GB to 450GB, and says there is no room! And there
isn't. What cud be causing this?
Thanks
JW

You're copying a folder with hard links or Junction Points in it.

Some files are getting copied twice. Or, File Explorer refused
to descend the Junction Point and see the files present in there.
That's why the size was on the low side, before you started the
copy,.

The contents of this copy just fine:

C:\users\Cowboy Wayne\Downloads

Whereas at this level, there are multiple Junction Points
which add all sorts of complexity to "backup copies". You'll
need to be a level 39 wizard to do this *properly*.

C:\users\Cowboy Wayne\

At some point, you should be making Macrium Reflect Free images,
because it never fouls up. It knows all this stuff. And won't
be fooled. Macrium Reflect Free images can be "mounted" later
like a hard drive, and you can random-access files and copy
them to the real C drive when you need them. There is no need
to "restore" the entire image, to get a file you want later.

File Explorer, by comparison, is a piece of crap.

*******

I didn't know this a month ago. I tried to do properties
on the

C:\users\Cowboy Wayne\

level, and it said there were 3GB of files. When I did
properties on

C:\users\Cowboy Wayne\Downloads

level, it said there were 67GB of files there. (A total
of an actual 70GB for the entire home directory tree.)

Obviously, if I try and copy something with File Explorer,
where it refuses to descend a Junction Point, I'm going
to be missing stuff.

Using the Sysinternals.com "junction" program...

https://docs.microsoft.com/en-us/sys...loads/junction

junction -s c:\

it will list around 60 or so Junction Points, some of
which are in C:\users\Cowboy Wayne\ . There is the
value you see, and the value underneath. You can test an
individual absolute path if you want, and see if it is a
junction or not.

junction C:\users\Cowboy Wayne\Downloads

The value underneath would get copied, as far as I know
(that's if it is seen, off to the side on its own). But when
calculating size by doing properties, File Explorer won't
descend Junctions or do appropriate math. This requires an
extraordinary effort from the computer user.

I studied the home directory, after the 17763 "data loss"
incident. I was trying to see if they deleted any of my
files. And that's when I discovered my home directory

C:\users\Cowboy Wayne\

level, was a virtual *mine field*. It takes a lot of
thought and planning, to get a good result there. Dragging
and dropping, requires planning. I wanted to automate
stuff, but it was pretty damn difficult to deal with.

It's not really your home directory - the OS has usurped
parts of it. This is one reason I keep all my goods in
the Download folder.

*******

I may not have explained it well enough for you.

All I can do is warn you "folder copy people" out there,
that you are *not* backing up your files properly! All
you guys copying random sections of your home directory
to a USB stick, could easily miss stuff.

You're missing stuff.

And I wasn't aware of this either, up to about a month
ago, when I started looking at it.

Once you've done a thorough examination of the home
directory, you'll begin to appreciate it's filled
with alligators.

Just do a Properties on C:\users\Cowboy Wayne\
and see how the size isn't even remotely close to
the right value. I was shocked when mine said 3GB,
and then when I checked my Downloads, it was 67GB.
Then I knew I didn't know how it worked.

Paul

On Mon, 29 Oct 2018 21:28:30 -0400, Paul
wrote:

wrote:
This seems crazy to me.
I am trying to copy-paste a large folder (291GB) from C drive to G
drive (external). I have done this before with no problems. When I
do a 'Properties' check' of the folder on C-drive folder it says
291GB, When I do the same of the folder on G-drive it says 17GB.
There is room for 291GB on G-drive. When I try a copy-paste it
pre-counts past 291GB to 450GB, and says there is no room! And there
isn't. What cud be causing this?
Thanks
JW

You're copying a folder with hard links or Junction Points in it.

Some files are getting copied twice. Or, File Explorer refused
to descend the Junction Point and see the files present in there.
That's why the size was on the low side, before you started the
copy,.

The contents of this copy just fine:

C:\users\Cowboy Wayne\Downloads

Whereas at this level, there are multiple Junction Points
which add all sorts of complexity to "backup copies". You'll
need to be a level 39 wizard to do this *properly*.

C:\users\Cowboy Wayne\

At some point, you should be making Macrium Reflect Free images,
because it never fouls up. It knows all this stuff. And won't
be fooled. Macrium Reflect Free images can be "mounted" later
like a hard drive, and you can random-access files and copy
them to the real C drive when you need them. There is no need
to "restore" the entire image, to get a file you want later.

File Explorer, by comparison, is a piece of crap.

*******

I didn't know this a month ago. I tried to do properties
on the

C:\users\Cowboy Wayne\

level, and it said there were 3GB of files. When I did
properties on

C:\users\Cowboy Wayne\Downloads

level, it said there were 67GB of files there. (A total
of an actual 70GB for the entire home directory tree.)

Obviously, if I try and copy something with File Explorer,
where it refuses to descend a Junction Point, I'm going
to be missing stuff.

Using the Sysinternals.com "junction" program...

https://docs.microsoft.com/en-us/sys...loads/junction

junction -s c:\

it will list around 60 or so Junction Points, some of
which are in C:\users\Cowboy Wayne\ . There is the
value you see, and the value underneath. You can test an
individual absolute path if you want, and see if it is a
junction or not.

junction C:\users\Cowboy Wayne\Downloads

The value underneath would get copied, as far as I know
(that's if it is seen, off to the side on its own). But when
calculating size by doing properties, File Explorer won't
descend Junctions or do appropriate math. This requires an
extraordinary effort from the computer user.

I studied the home directory, after the 17763 "data loss"
incident. I was trying to see if they deleted any of my
files. And that's when I discovered my home directory

C:\users\Cowboy Wayne\

level, was a virtual *mine field*. It takes a lot of
thought and planning, to get a good result there. Dragging
and dropping, requires planning. I wanted to automate
stuff, but it was pretty damn difficult to deal with.

It's not really your home directory - the OS has usurped
parts of it. This is one reason I keep all my goods in
the Download folder.

*******

I may not have explained it well enough for you.

All I can do is warn you "folder copy people" out there,
that you are *not* backing up your files properly! All
you guys copying random sections of your home directory
to a USB stick, could easily miss stuff.

You're missing stuff.

And I wasn't aware of this either, up to about a month
ago, when I started looking at it.

Once you've done a thorough examination of the home
directory, you'll begin to appreciate it's filled
with alligators.

Just do a Properties on C:\users\Cowboy Wayne\
and see how the size isn't even remotely close to
the right value. I was shocked when mine said 3GB,
and then when I checked my Downloads, it was 67GB.
Then I knew I didn't know how it worked.

Paul


Wow! Live and learn!
I never knew this.
Thanks
JW

On Mon, 29 Oct 2018 21:28:30 -0400, Paul
wrote:

wrote:
This seems crazy to me.
I am trying to copy-paste a large folder (291GB) from C drive to G
drive (external). I have done this before with no problems. When I
do a 'Properties' check' of the folder on C-drive folder it says
291GB, When I do the same of the folder on G-drive it says 17GB.
There is room for 291GB on G-drive. When I try a copy-paste it
pre-counts past 291GB to 450GB, and says there is no room! And there
isn't. What cud be causing this?
Thanks
JW

You're copying a folder with hard links or Junction Points in it.

Some files are getting copied twice. Or, File Explorer refused
to descend the Junction Point and see the files present in there.
That's why the size was on the low side, before you started the
copy,.

The contents of this copy just fine:

C:\users\Cowboy Wayne\Downloads

Whereas at this level, there are multiple Junction Points
which add all sorts of complexity to "backup copies". You'll
need to be a level 39 wizard to do this *properly*.

C:\users\Cowboy Wayne\

At some point, you should be making Macrium Reflect Free images,
because it never fouls up. It knows all this stuff. And won't
be fooled. Macrium Reflect Free images can be "mounted" later
like a hard drive, and you can random-access files and copy
them to the real C drive when you need them. There is no need
to "restore" the entire image, to get a file you want later.

File Explorer, by comparison, is a piece of crap.

*******

I didn't know this a month ago. I tried to do properties
on the

C:\users\Cowboy Wayne\

level, and it said there were 3GB of files. When I did
properties on

C:\users\Cowboy Wayne\Downloads

level, it said there were 67GB of files there. (A total
of an actual 70GB for the entire home directory tree.)

Obviously, if I try and copy something with File Explorer,
where it refuses to descend a Junction Point, I'm going
to be missing stuff.

Using the Sysinternals.com "junction" program...

https://docs.microsoft.com/en-us/sys...loads/junction

junction -s c:\

it will list around 60 or so Junction Points, some of
which are in C:\users\Cowboy Wayne\ . There is the
value you see, and the value underneath. You can test an
individual absolute path if you want, and see if it is a
junction or not.

junction C:\users\Cowboy Wayne\Downloads

The value underneath would get copied, as far as I know
(that's if it is seen, off to the side on its own). But when
calculating size by doing properties, File Explorer won't
descend Junctions or do appropriate math. This requires an
extraordinary effort from the computer user.

I studied the home directory, after the 17763 "data loss"
incident. I was trying to see if they deleted any of my
files. And that's when I discovered my home directory

C:\users\Cowboy Wayne\

level, was a virtual *mine field*. It takes a lot of
thought and planning, to get a good result there. Dragging
and dropping, requires planning. I wanted to automate
stuff, but it was pretty damn difficult to deal with.

It's not really your home directory - the OS has usurped
parts of it. This is one reason I keep all my goods in
the Download folder.

*******

I may not have explained it well enough for you.

All I can do is warn you "folder copy people" out there,
that you are *not* backing up your files properly! All
you guys copying random sections of your home directory
to a USB stick, could easily miss stuff.

You're missing stuff.

And I wasn't aware of this either, up to about a month
ago, when I started looking at it.

Once you've done a thorough examination of the home
directory, you'll begin to appreciate it's filled
with alligators.

Just do a Properties on C:\users\Cowboy Wayne\
and see how the size isn't even remotely close to
the right value. I was shocked when mine said 3GB,
and then when I checked my Downloads, it was 67GB.
Then I knew I didn't know how it worked.

Paul


What the H.
I just brought up my faithful Macrium V7, and it only shows my two
data drives- D and G. Where did C drive go? That's the drive by the
way that is showing the erroneous file/folder sizes,
Help
JW

On Tue, 30 Oct 2018 06:34:05 -0400, wrote:

On Mon, 29 Oct 2018 21:28:30 -0400, Paul
wrote:

wrote:
This seems crazy to me.
I am trying to copy-paste a large folder (291GB) from C drive to G
drive (external). I have done this before with no problems. When I
do a 'Properties' check' of the folder on C-drive folder it says
291GB, When I do the same of the folder on G-drive it says 17GB.
There is room for 291GB on G-drive. When I try a copy-paste it
pre-counts past 291GB to 450GB, and says there is no room! And there
isn't. What cud be causing this?
Thanks
JW

You're copying a folder with hard links or Junction Points in it.

Some files are getting copied twice. Or, File Explorer refused
to descend the Junction Point and see the files present in there.
That's why the size was on the low side, before you started the
copy,.

The contents of this copy just fine:

C:\users\Cowboy Wayne\Downloads

Whereas at this level, there are multiple Junction Points
which add all sorts of complexity to "backup copies". You'll
need to be a level 39 wizard to do this *properly*.

C:\users\Cowboy Wayne\

At some point, you should be making Macrium Reflect Free images,
because it never fouls up. It knows all this stuff. And won't
be fooled. Macrium Reflect Free images can be "mounted" later
like a hard drive, and you can random-access files and copy
them to the real C drive when you need them. There is no need
to "restore" the entire image, to get a file you want later.

File Explorer, by comparison, is a piece of crap.

*******

I didn't know this a month ago. I tried to do properties
on the

C:\users\Cowboy Wayne\

level, and it said there were 3GB of files. When I did
properties on

C:\users\Cowboy Wayne\Downloads

level, it said there were 67GB of files there. (A total
of an actual 70GB for the entire home directory tree.)

Obviously, if I try and copy something with File Explorer,
where it refuses to descend a Junction Point, I'm going
to be missing stuff.

Using the Sysinternals.com "junction" program...

https://docs.microsoft.com/en-us/sys...loads/junction

junction -s c:\

it will list around 60 or so Junction Points, some of
which are in C:\users\Cowboy Wayne\ . There is the
value you see, and the value underneath. You can test an
individual absolute path if you want, and see if it is a
junction or not.

junction C:\users\Cowboy Wayne\Downloads

The value underneath would get copied, as far as I know
(that's if it is seen, off to the side on its own). But when
calculating size by doing properties, File Explorer won't
descend Junctions or do appropriate math. This requires an
extraordinary effort from the computer user.

I studied the home directory, after the 17763 "data loss"
incident. I was trying to see if they deleted any of my
files. And that's when I discovered my home directory

C:\users\Cowboy Wayne\

level, was a virtual *mine field*. It takes a lot of
thought and planning, to get a good result there. Dragging
and dropping, requires planning. I wanted to automate
stuff, but it was pretty damn difficult to deal with.

It's not really your home directory - the OS has usurped
parts of it. This is one reason I keep all my goods in
the Download folder.

*******

I may not have explained it well enough for you.

All I can do is warn you "folder copy people" out there,
that you are *not* backing up your files properly! All
you guys copying random sections of your home directory
to a USB stick, could easily miss stuff.

You're missing stuff.

And I wasn't aware of this either, up to about a month
ago, when I started looking at it.

Once you've done a thorough examination of the home
directory, you'll begin to appreciate it's filled
with alligators.

Just do a Properties on C:\users\Cowboy Wayne\
and see how the size isn't even remotely close to
the right value. I was shocked when mine said 3GB,
and then when I checked my Downloads, it was 67GB.
Then I knew I didn't know how it worked.

Paul


What the H.
I just brought up my faithful Macrium V7, and it only shows my two
data drives- D and G. Where did C drive go? That's the drive by the
way that is showing the erroneous file/folder sizes,
Help
JW
Never mind. Dumb me. I found it.
JW

wrote:

What the H.
I just brought up my faithful Macrium V7, and it only shows my two
data drives- D and G. Where did C drive go? That's the drive by the
way that is showing the erroneous file/folder sizes,
Help
JW
Never mind. Dumb me. I found it.
JW

I just want to make sure your data is copied... somewhere.

I never suspected an issue (because I don't work at the
root level of the home directory all that often).

I always work in my Downloads folder :-)

Paul

On Tue, 30 Oct 2018 07:52:57 -0400, Paul
wrote:

wrote:

What the H.
I just brought up my faithful Macrium V7, and it only shows my two
data drives- D and G. Where did C drive go? That's the drive by the
way that is showing the erroneous file/folder sizes,
Help
JW
Never mind. Dumb me. I found it.
JW

I just want to make sure your data is copied... somewhere.

I never suspected an issue (because I don't work at the
root level of the home directory all that often).

I always work in my Downloads folder :-)

Paul

It seems quite irrational to me that W10 will tell me via Windows
Explorer that I have a folder containing some 300GB one place, and
then tell me in another place that the same folder requires some 500GB
of space somewhere else. Its just not possible I'm thinkin'. At the
very least it is very worrisome - I don't really know any more what I
have. I still think I must be doing something wrong.
JW

On Tue, 30 Oct 2018 13:44:30 -0400, wrote:

It seems quite irrational to me that W10 will tell me via Windows
Explorer that I have a folder containing some 300GB one place, and
then tell me in another place that the same folder requires some 500GB
of space somewhere else. Its just not possible I'm thinkin'. At the
very least it is very worrisome - I don't really know any more what I
have. I still think I must be doing something wrong.

If the folder in question is a Windows system folder, i.e., a folder
whose contents are solely managed by Windows itself, then I'm not seeing
any reason for concern.

OTOH, if this is one of your own folders, one that you've created and
put your own files into it, and assuming you didn't play around with
junction points, then I'd be very concerned because it quite possibly
indicates that something has gone seriously wrong.

If you're very paranoid, create a disk image and work from that.
Otherwise, consider working at the file level rather than the folder
level and start moving files out of there. All will eventually be
revealed.

On Tue, 30 Oct 2018 14:37:55 -0500, Char Jackson
wrote:

On Tue, 30 Oct 2018 13:44:30 -0400, wrote:

It seems quite irrational to me that W10 will tell me via Windows
Explorer that I have a folder containing some 300GB one place, and
then tell me in another place that the same folder requires some 500GB
of space somewhere else. Its just not possible I'm thinkin'. At the
very least it is very worrisome - I don't really know any more what I
have. I still think I must be doing something wrong.

If the folder in question is a Windows system folder, i.e., a folder
whose contents are solely managed by Windows itself, then I'm not seeing
any reason for concern.

OTOH, if this is one of your own folders, one that you've created and
put your own files into it, and assuming you didn't play around with
junction points, then I'd be very concerned because it quite possibly
indicates that something has gone seriously wrong.

If you're very paranoid, create a disk image and work from that.
Otherwise, consider working at the file level rather than the folder
level and start moving files out of there. All will eventually be
revealed.
Thanx for response.
It is a whole bunch of files in a whole bunch of folders which are
mostly data for my diffferent c-drive programs. My intent was to keep
data on a separate, external drive activated only when needed. Seemed
like a good idea at the time. Never had a problem until the external
drive acted up, and I decided to try to back it up with copy/pastes to
another drive. That's when the size anomalies showed up and shocked
the H out of me. I still do not understand it. Size is size after
all. I have my data back on c-drive (from a Macrium image) now, and
when I have its dupe back on an external drive, it will be both
places. That's where I am running into the size problem. Then I
have to decide which to expose and use, or even if to do it. .
W10?
JW

Char Jackson wrote:
On Tue, 30 Oct 2018 13:44:30 -0400, wrote:

It seems quite irrational to me that W10 will tell me via Windows
Explorer that I have a folder containing some 300GB one place, and
then tell me in another place that the same folder requires some 500GB
of space somewhere else. Its just not possible I'm thinkin'. At the
very least it is very worrisome - I don't really know any more what I
have. I still think I must be doing something wrong.

If the folder in question is a Windows system folder, i.e., a folder
whose contents are solely managed by Windows itself, then I'm not seeing
any reason for concern.

OTOH, if this is one of your own folders, one that you've created and
put your own files into it, and assuming you didn't play around with
junction points, then I'd be very concerned because it quite possibly
indicates that something has gone seriously wrong.

If you're very paranoid, create a disk image and work from that.
Otherwise, consider working at the file level rather than the folder
level and start moving files out of there. All will eventually be
revealed.


File Explorer does not seem able to descend Junction Points.

icacls cannot descend junction points.

When Robocopy descends a junction point, the path becomes
longer and longer until the tool hits "path too long" and
the transfer stops. Other tools do this too, except the
diagnostic you might get from Robocopy, might show the
path it constructed before blowing up.

Robocopy has a command line option to "step over"
Junction Points and not process them at all. This causes
a transfer to complete, followed by the user doing the math
to figure out what got missed.

*******

Where the junction point points to, is sometimes a directory
at the same level ("parallel") to the Junction Point, so other
tools still "pick up" the thing in question. This can deceive
a user into thinking they're "still on WinXP".

And when a Junction Point is redirected to another volume,
the user is supposed to know that copying will not go
off-volume to capture stuff. If you move your

C:\users\user name\Downloads

to

D:\Downloads

using a Junction Point method, if you drag and drop

C:\users\user name

onto your USB stock, you're likely *not* getting a copy
of D:\Downloads on the USB stick.

It really depends what tools handle Junction Points
they way you would expect.

On Unix, you could have soft links and get reasonable
things to happen. When you wanted to prevent Unix
from going off-volume on a "lark", you used the -xdev
argument to stop it. Windows just doesn't seem to have
the necessary documentation and warnings for the
behaviors, whatever they are.

That's why you have to think carefully about what's
in that folder, what it points to, and whether you
can "reasonably expect" your naive drag and drop to
actually contain what you expected.

Whoever invented Junction Points... was a demented genius :-)
Personal opinion of course.

Maybe someone who knows all the behaviors better,
could write up a description. All I know is

"Danger, Will Robinson"

I do know my Macrium Reflect backup images have
everything, and those are what I count on to have
copies of stuff. Where the size might not match
on Macrium, is I can never be sure whether it
picks up Pagefile or Hiberfile. You may notice
differences as well on Macrium, between running
Macrium from C: and running it from the CD, as the
output size is slightly different. That could be
pagefile/hiberfile, or it could be related to
VSS shadow behavior (what to do with shadows in System
Volume Information).

Paul

On Tue, 30 Oct 2018 19:44:08 -0400, Paul
wrote:

Char Jackson wrote:
On Tue, 30 Oct 2018 13:44:30 -0400, wrote:

It seems quite irrational to me that W10 will tell me via Windows
Explorer that I have a folder containing some 300GB one place, and
then tell me in another place that the same folder requires some 500GB
of space somewhere else. Its just not possible I'm thinkin'. At the
very least it is very worrisome - I don't really know any more what I
have. I still think I must be doing something wrong.

If the folder in question is a Windows system folder, i.e., a folder
whose contents are solely managed by Windows itself, then I'm not seeing
any reason for concern.

OTOH, if this is one of your own folders, one that you've created and
put your own files into it, and assuming you didn't play around with
junction points, then I'd be very concerned because it quite possibly
indicates that something has gone seriously wrong.

If you're very paranoid, create a disk image and work from that.
Otherwise, consider working at the file level rather than the folder
level and start moving files out of there. All will eventually be
revealed.


File Explorer does not seem able to descend Junction Points.

icacls cannot descend junction points.

When Robocopy descends a junction point, the path becomes
longer and longer until the tool hits "path too long" and
the transfer stops. Other tools do this too, except the
diagnostic you might get from Robocopy, might show the
path it constructed before blowing up.

Robocopy has a command line option to "step over"
Junction Points and not process them at all. This causes
a transfer to complete, followed by the user doing the math
to figure out what got missed.

*******

Where the junction point points to, is sometimes a directory
at the same level ("parallel") to the Junction Point, so other
tools still "pick up" the thing in question. This can deceive
a user into thinking they're "still on WinXP".

And when a Junction Point is redirected to another volume,
the user is supposed to know that copying will not go
off-volume to capture stuff. If you move your

C:\users\user name\Downloads

to

D:\Downloads

using a Junction Point method, if you drag and drop

C:\users\user name

onto your USB stock, you're likely *not* getting a copy
of D:\Downloads on the USB stick.

It really depends what tools handle Junction Points
they way you would expect.

On Unix, you could have soft links and get reasonable
things to happen. When you wanted to prevent Unix
from going off-volume on a "lark", you used the -xdev
argument to stop it. Windows just doesn't seem to have
the necessary documentation and warnings for the
behaviors, whatever they are.

That's why you have to think carefully about what's
in that folder, what it points to, and whether you
can "reasonably expect" your naive drag and drop to
actually contain what you expected.

Whoever invented Junction Points... was a demented genius :-)
Personal opinion of course.

Maybe someone who knows all the behaviors better,
could write up a description. All I know is

"Danger, Will Robinson"

I do know my Macrium Reflect backup images have
everything, and those are what I count on to have
copies of stuff. Where the size might not match
on Macrium, is I can never be sure whether it
picks up Pagefile or Hiberfile. You may notice
differences as well on Macrium, between running
Macrium from C: and running it from the CD, as the
output size is slightly different. That could be
pagefile/hiberfile, or it could be related to
VSS shadow behavior (what to do with shadows in System
Volume Information).

Paul

Well - I sure thought I had a good idea. I'll just continue to do
Macrium backups/restores. I opened a can of worms for sure.
Bye, and thanks for help.
JW