Robocopy Not Terminating

On 12/1/2017 6:11 PM, Paul wrote:
I just tested your version of Robocopy, on a folder with
two 18.6GB files, transferred to a second disk drive. And
both copied no problem at all.

Is the Robocopy.exe in 64-bit Windows also a 64-bit program?

--
@~@ Remain silent! Drink, Blink, Stretch! Live long and prosper!!
/ v \ Simplicity is Beauty!
/( _ )\ May the Force and farces be with you!
^ ^ (x86_64 Ubuntu 9.10) Linux 2.6.39.3
�借貸! ��騙! ��交! �打交! �打劫! �自殺! 請考慮綜� (CSSA):
http://www.swd.gov.hk/tc/index/site_...sub_addressesa

On 1/20/2017 10:04 AM, Mr. Man-wai Chang wrote:
On 12/1/2017 2:37 PM, David E. Ross wrote:
For files of size 1.5 GB or less, Robocopy.exe terminates, allowing the
script to continue with additional commands. For files of size 10 GB or
more, Robocopy.exe does NOT terminate, preventing the script to
continue. The same script either has the problem (file larger than 10
GB) or does not have the problem (file less than 1.5 GB).

I am not sure where between 1.5 GB and 10 GB the problem arises.

Robocopy is a 32-bit program.

I think the maximum size of file should be limited to 2 GB?

Need to port and compile the program using 64-bit C/C++ compilers?

For the meantime, use a 64-bit 7-zip to compress and split the 10GB
files into multiple 1.5GB files?


Win 7 Home 64 bit

Robocopy 6.1.7601.23403

I have a batch using Robocopy from NAS over CAT 5 to PC to USB3 1 TB HDD
to backup an Acronis Images files (tib) 500GB (full) and 30 to 50 GB
(incremental) without problems except the time it takes to finish
writing the larger file.

Here are my central batch cmds:
SET "ipNAS=\\xxx.xxx.x.xx"
SET "_Set0=%ipNAS%\Image\i7\Set0"
SET _LOG=/LOG:F:\BackupSet0.txt
SET _ACT=/PURGE /R:0 /W:0 /NDL /NFL %_LOG%
START "" /WAIT Robocopy %_Set0% K:\Images_Set0 %_ACT%

I suggest monitoring the file size being written to view what is
happening. This can be done using a batch loop with SET _Size=%%~zI

On Fri, 20 Jan 2017 23:04:41 +0800, "Mr. Man-wai Chang"
wrote:

On 12/1/2017 2:37 PM, David E. Ross wrote:
For files of size 1.5 GB or less, Robocopy.exe terminates, allowing the
script to continue with additional commands. For files of size 10 GB or
more, Robocopy.exe does NOT terminate, preventing the script to
continue. The same script either has the problem (file larger than 10
GB) or does not have the problem (file less than 1.5 GB).

I am not sure where between 1.5 GB and 10 GB the problem arises.

Robocopy is a 32-bit program.

I think the maximum size of file should be limited to 2 GB?

No. Even if you meant 4GB-1, still no.

Need to port and compile the program using 64-bit C/C++ compilers?

No.

For the meantime, use a 64-bit 7-zip to compress and split the 10GB
files into multiple 1.5GB files?

No.

--

Char Jackson

Mr. Man-wai Chang wrote:
On 12/1/2017 6:11 PM, Paul wrote:
I just tested your version of Robocopy, on a folder with
two 18.6GB files, transferred to a second disk drive. And
both copied no problem at all.

Is the Robocopy.exe in 64-bit Windows also a 64-bit program?


When the NTFS file system was invented, it *already* needed
to support a 64 bit file length field, even in 32 bit
executables. It's been doing that for at least 17 years.
We've been able to copy 4GB files in NTFS, since NTFS
came out! It doesn't matter whether the program is
32 bit or 64 bit, since we were able to write 32 bit
programs in the year 2000 that could copy large files.
The 32-bit nature of the program would be a red herring.

Since I already tested the exact version of Robocopy
as the OP is using, and transferred a 25GB file with it,
that proves that the program can handle something which
is 4GB.

In the old days (FAT32 era), you might have written
programs with stat(). From the year NTFS was invented,
it would have been wise to start using stat64(), which
exists even in 32-bit programs. Then, you can properly
handle files with sizes 4GB.

Paul

On 1/20/2017 9:09 AM, Zaidy036 wrote:
On 1/20/2017 10:04 AM, Mr. Man-wai Chang wrote:
On 12/1/2017 2:37 PM, David E. Ross wrote:
For files of size 1.5 GB or less, Robocopy.exe terminates, allowing the
script to continue with additional commands. For files of size 10 GB or
more, Robocopy.exe does NOT terminate, preventing the script to
continue. The same script either has the problem (file larger than 10
GB) or does not have the problem (file less than 1.5 GB).

I am not sure where between 1.5 GB and 10 GB the problem arises.

Robocopy is a 32-bit program.

I think the maximum size of file should be limited to 2 GB?

Need to port and compile the program using 64-bit C/C++ compilers?

For the meantime, use a 64-bit 7-zip to compress and split the 10GB
files into multiple 1.5GB files?


Win 7 Home 64 bit

Robocopy 6.1.7601.23403

I have a batch using Robocopy from NAS over CAT 5 to PC to USB3 1 TB HDD
to backup an Acronis Images files (tib) 500GB (full) and 30 to 50 GB
(incremental) without problems except the time it takes to finish
writing the larger file.

Here are my central batch cmds:
SET "ipNAS=\\xxx.xxx.x.xx"
SET "_Set0=%ipNAS%\Image\i7\Set0"
SET _LOG=/LOG:F:\BackupSet0.txt
SET _ACT=/PURGE /R:0 /W:0 /NDL /NFL %_LOG%
START "" /WAIT Robocopy %_Set0% K:\Images_Set0 %_ACT%

I suggest monitoring the file size being written to view what is
happening. This can be done using a batch loop with SET _Size=%%~zI

Where did you get Robocopy 6.1.7601.23403?


--
David E. Ross
http://www.rossde.com/

When the President of the United States makes a statement of
national importance, I want to see his face as he is talking.
At the least, I want to hear his voice.

Donald: Stop tweeting. Otherwise, how do we know the message
really comes from you?

On 1/21/2017 12:55 AM, David E. Ross wrote:
On 1/20/2017 9:09 AM, Zaidy036 wrote:
On 1/20/2017 10:04 AM, Mr. Man-wai Chang wrote:
On 12/1/2017 2:37 PM, David E. Ross wrote:
For files of size 1.5 GB or less, Robocopy.exe terminates, allowing the
script to continue with additional commands. For files of size 10 GB or
more, Robocopy.exe does NOT terminate, preventing the script to
continue. The same script either has the problem (file larger than 10
GB) or does not have the problem (file less than 1.5 GB).

I am not sure where between 1.5 GB and 10 GB the problem arises.

Robocopy is a 32-bit program.

I think the maximum size of file should be limited to 2 GB?

Need to port and compile the program using 64-bit C/C++ compilers?

For the meantime, use a 64-bit 7-zip to compress and split the 10GB
files into multiple 1.5GB files?


Win 7 Home 64 bit

Robocopy 6.1.7601.23403

I have a batch using Robocopy from NAS over CAT 5 to PC to USB3 1 TB HDD
to backup an Acronis Images files (tib) 500GB (full) and 30 to 50 GB
(incremental) without problems except the time it takes to finish
writing the larger file.

Here are my central batch cmds:
SET "ipNAS=\\xxx.xxx.x.xx"
SET "_Set0=%ipNAS%\Image\i7\Set0"
SET _LOG=/LOG:F:\BackupSet0.txt
SET _ACT=/PURGE /R:0 /W:0 /NDL /NFL %_LOG%
START "" /WAIT Robocopy %_Set0% K:\Images_Set0 %_ACT%

I suggest monitoring the file size being written to view what is
happening. This can be done using a batch loop with SET _Size=%%~zI

Where did you get Robocopy 6.1.7601.23403?



"C:\Windows\System32\Robocopy.exe" Properties

On 1/21/2017 7:46 AM, Zaidy036 wrote:
On 1/21/2017 12:55 AM, David E. Ross wrote:
On 1/20/2017 9:09 AM, Zaidy036 wrote:

Win 7 Home 64 bit

Robocopy 6.1.7601.23403

I have a batch using Robocopy from NAS over CAT 5 to PC to USB3 1 TB HDD
to backup an Acronis Images files (tib) 500GB (full) and 30 to 50 GB
(incremental) without problems except the time it takes to finish
writing the larger file.

Here are my central batch cmds:
SET "ipNAS=\\xxx.xxx.x.xx"
SET "_Set0=%ipNAS%\Image\i7\Set0"
SET _LOG=/LOG:F:\BackupSet0.txt
SET _ACT=/PURGE /R:0 /W:0 /NDL /NFL %_LOG%
START "" /WAIT Robocopy %_Set0% K:\Images_Set0 %_ACT%

I suggest monitoring the file size being written to view what is
happening. This can be done using a batch loop with SET _Size=%%~zI

Where did you get Robocopy 6.1.7601.23403?



"C:\Windows\System32\Robocopy.exe" Properties

At that location, I have Robocopy.exe 5.1.10.1027, last modified 20
November 2010.

--
David E. Ross
http://www.rossde.com/

When the President of the United States makes a statement of
national importance, I want to see his face as he is talking.
At the least, I want to hear his voice.

Donald: Stop tweeting. Otherwise, how do we know the message
really comes from you?

On 1/21/2017 10:46 AM, Zaidy036 wrote:
On 1/21/2017 12:55 AM, David E. Ross wrote:
On 1/20/2017 9:09 AM, Zaidy036 wrote:
On 1/20/2017 10:04 AM, Mr. Man-wai Chang wrote:
On 12/1/2017 2:37 PM, David E. Ross wrote:
For files of size 1.5 GB or less, Robocopy.exe terminates, allowing
the
script to continue with additional commands. For files of size 10
GB or
more, Robocopy.exe does NOT terminate, preventing the script to
continue. The same script either has the problem (file larger than 10
GB) or does not have the problem (file less than 1.5 GB).

I am not sure where between 1.5 GB and 10 GB the problem arises.

Robocopy is a 32-bit program.

I think the maximum size of file should be limited to 2 GB?

Need to port and compile the program using 64-bit C/C++ compilers?

For the meantime, use a 64-bit 7-zip to compress and split the 10GB
files into multiple 1.5GB files?


Win 7 Home 64 bit

Robocopy 6.1.7601.23403

I have a batch using Robocopy from NAS over CAT 5 to PC to USB3 1 TB HDD
to backup an Acronis Images files (tib) 500GB (full) and 30 to 50 GB
(incremental) without problems except the time it takes to finish
writing the larger file.

Here are my central batch cmds:
SET "ipNAS=\\xxx.xxx.x.xx"
SET "_Set0=%ipNAS%\Image\i7\Set0"
SET _LOG=/LOG:F:\BackupSet0.txt
SET _ACT=/PURGE /R:0 /W:0 /NDL /NFL %_LOG%
START "" /WAIT Robocopy %_Set0% K:\Images_Set0 %_ACT%

I suggest monitoring the file size being written to view what is
happening. This can be done using a batch loop with SET _Size=%%~zI

Where did you get Robocopy 6.1.7601.23403?



"C:\Windows\System32\Robocopy.exe" Properties

Just did an Everything search for RoboCopy.exe "Executable:"

5.1.10.1027
"C:\Windows\winsxs\amd64_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.17514_none_252d 34f00303c6fa\Robocopy.exe"

6.1.7601.23403
"C:\Windows\winsxs\amd64_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.23403_none_25c0 70011c1a6a70\Robocopy.exe"

5.1.10.1027
"C:\Windows\winsxs\x86_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.17514_none_c90e 996c4aa655c4\Robocopy.exe"

6.1.7601.23403
"C:\Windows\winsxs\x86_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.23403_none_c9a1 d47d63bcf93a\Robocopy.exe"

Two are 5.1.10.1027 and two 6.1.7601.23403 **BUT** why does the list
**NOT** include "C:\Windows\System32\Robocopy.exe" ???

Nothing in the RoboCopy Log indicates which version was being used but
since CMD comes up using C:\Windows\System32 I expect
"C:\Windows\System32\Robocopy.exe".

Zaidy036 wrote:
On 1/21/2017 12:55 AM, David E. Ross wrote:
On 1/20/2017 9:09 AM, Zaidy036 wrote:
On 1/20/2017 10:04 AM, Mr. Man-wai Chang wrote:
On 12/1/2017 2:37 PM, David E. Ross wrote:
For files of size 1.5 GB or less, Robocopy.exe terminates, allowing
the
script to continue with additional commands. For files of size 10
GB or
more, Robocopy.exe does NOT terminate, preventing the script to
continue. The same script either has the problem (file larger than 10
GB) or does not have the problem (file less than 1.5 GB).

I am not sure where between 1.5 GB and 10 GB the problem arises.

Robocopy is a 32-bit program.

I think the maximum size of file should be limited to 2 GB?

Need to port and compile the program using 64-bit C/C++ compilers?

For the meantime, use a 64-bit 7-zip to compress and split the 10GB
files into multiple 1.5GB files?


Win 7 Home 64 bit

Robocopy 6.1.7601.23403

I have a batch using Robocopy from NAS over CAT 5 to PC to USB3 1 TB HDD
to backup an Acronis Images files (tib) 500GB (full) and 30 to 50 GB
(incremental) without problems except the time it takes to finish
writing the larger file.

Here are my central batch cmds:
SET "ipNAS=\\xxx.xxx.x.xx"
SET "_Set0=%ipNAS%\Image\i7\Set0"
SET _LOG=/LOG:F:\BackupSet0.txt
SET _ACT=/PURGE /R:0 /W:0 /NDL /NFL %_LOG%
START "" /WAIT Robocopy %_Set0% K:\Images_Set0 %_ACT%

I suggest monitoring the file size being written to view what is
happening. This can be done using a batch loop with SET _Size=%%~zI

Where did you get Robocopy 6.1.7601.23403?



"C:\Windows\System32\Robocopy.exe" Properties


https://blogs.technet.microsoft.com/...you-kb3125574/

"Windows 7 SP1 and Server 2008 R2 SP1 convenience roll-up
now available at a download location near you! (KB3125574)"

I took a look in my copy of that download. Because I use this when
testing Windows Update for Win7SP1 (in a test VM, not on my real copy).

AMD64-all-windows6.1-kb3125574-v4-x64_2dafb1d203c8964239af3048b5dd4b1264cd93b9.msu

500,046,015 bytes

You can open "_manifest_.cix.xml" in the archive. For example,
this entry looks interesting.

File id="45586" name="x86_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.23403_none_c9a1 d47d63bcf93a\robocopy.exe" length="101888" time="131034005737750000" attr="32"
Hash alg="SHA1" value="903fff4b49ee8f38c18af4573ecdad636890a2ed" /
Delta
Source type="PA30" name="11086"
Hash alg="SHA1" value="856973479dd460752613c113d2e6439c1b8719de" /
/Source
Basis file="32850" /
/Delta
/File

Now, with that information, I'm not able to say anything.
I couldn't find any of the referenced material.

So what a person could do, is

1) visit catalog.update.microsoft.com and get a copy of kb3125574
2) Backup the OS.
3) Install it.
4) Grab the essential materials (EXE files and MUI maybe ?)
5) Restore from backup.

That's if there was actually something good about that version.

The convenience rollup is supposed to be made from
existing patches. So I don't know why the release number
of the robocopy.exe executable would be changed, just for
the rollup. I'm not able to find any other reference to that
particular one.

Paul

On 1/21/2017 11:36 AM, Paul wrote:
Zaidy036 wrote:
On 1/21/2017 12:55 AM, David E. Ross wrote:
On 1/20/2017 9:09 AM, Zaidy036 wrote:
On 1/20/2017 10:04 AM, Mr. Man-wai Chang wrote:
On 12/1/2017 2:37 PM, David E. Ross wrote:
For files of size 1.5 GB or less, Robocopy.exe terminates,
allowing the
script to continue with additional commands. For files of size 10
GB or
more, Robocopy.exe does NOT terminate, preventing the script to
continue. The same script either has the problem (file larger
than 10
GB) or does not have the problem (file less than 1.5 GB).

I am not sure where between 1.5 GB and 10 GB the problem arises.

Robocopy is a 32-bit program.

I think the maximum size of file should be limited to 2 GB?

Need to port and compile the program using 64-bit C/C++ compilers?

For the meantime, use a 64-bit 7-zip to compress and split the 10GB
files into multiple 1.5GB files?


Win 7 Home 64 bit

Robocopy 6.1.7601.23403

I have a batch using Robocopy from NAS over CAT 5 to PC to USB3 1 TB
HDD
to backup an Acronis Images files (tib) 500GB (full) and 30 to 50 GB
(incremental) without problems except the time it takes to finish
writing the larger file.

Here are my central batch cmds:
SET "ipNAS=\\xxx.xxx.x.xx"
SET "_Set0=%ipNAS%\Image\i7\Set0"
SET _LOG=/LOG:F:\BackupSet0.txt
SET _ACT=/PURGE /R:0 /W:0 /NDL /NFL %_LOG%
START "" /WAIT Robocopy %_Set0% K:\Images_Set0 %_ACT%

I suggest monitoring the file size being written to view what is
happening. This can be done using a batch loop with SET _Size=%%~zI

Where did you get Robocopy 6.1.7601.23403?



"C:\Windows\System32\Robocopy.exe" Properties


https://blogs.technet.microsoft.com/...you-kb3125574/


"Windows 7 SP1 and Server 2008 R2 SP1 convenience roll-up
now available at a download location near you! (KB3125574)"

I took a look in my copy of that download. Because I use this when
testing Windows Update for Win7SP1 (in a test VM, not on my real copy).


AMD64-all-windows6.1-kb3125574-v4-x64_2dafb1d203c8964239af3048b5dd4b1264cd93b9.msu


500,046,015 bytes

You can open "_manifest_.cix.xml" in the archive. For example,
this entry looks interesting.

File id="45586"
name="x86_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.23403_none_c9a1 d47d63bcf93a\robocopy.exe"
length="101888" time="131034005737750000" attr="32"
Hash alg="SHA1"
value="903fff4b49ee8f38c18af4573ecdad636890a2ed" /
Delta
Source type="PA30" name="11086"
Hash alg="SHA1"
value="856973479dd460752613c113d2e6439c1b8719de" /
/Source
Basis file="32850" /
/Delta
/File

Now, with that information, I'm not able to say anything.
I couldn't find any of the referenced material.

So what a person could do, is

1) visit catalog.update.microsoft.com and get a copy of kb3125574
2) Backup the OS.
3) Install it.
4) Grab the essential materials (EXE files and MUI maybe ?)
5) Restore from backup.

That's if there was actually something good about that version.

The convenience rollup is supposed to be made from
existing patches. So I don't know why the release number
of the robocopy.exe executable would be changed, just for
the rollup. I'm not able to find any other reference to that
particular one.

Paul

kb3125574 is installed on my PC

Zaidy036 wrote:
On 1/21/2017 10:46 AM, Zaidy036 wrote:
On 1/21/2017 12:55 AM, David E. Ross wrote:
On 1/20/2017 9:09 AM, Zaidy036 wrote:
On 1/20/2017 10:04 AM, Mr. Man-wai Chang wrote:
On 12/1/2017 2:37 PM, David E. Ross wrote:
For files of size 1.5 GB or less, Robocopy.exe terminates, allowing
the
script to continue with additional commands. For files of size 10
GB or
more, Robocopy.exe does NOT terminate, preventing the script to
continue. The same script either has the problem (file larger
than 10
GB) or does not have the problem (file less than 1.5 GB).

I am not sure where between 1.5 GB and 10 GB the problem arises.

Robocopy is a 32-bit program.

I think the maximum size of file should be limited to 2 GB?

Need to port and compile the program using 64-bit C/C++ compilers?

For the meantime, use a 64-bit 7-zip to compress and split the 10GB
files into multiple 1.5GB files?


Win 7 Home 64 bit

Robocopy 6.1.7601.23403

I have a batch using Robocopy from NAS over CAT 5 to PC to USB3 1 TB
HDD
to backup an Acronis Images files (tib) 500GB (full) and 30 to 50 GB
(incremental) without problems except the time it takes to finish
writing the larger file.

Here are my central batch cmds:
SET "ipNAS=\\xxx.xxx.x.xx"
SET "_Set0=%ipNAS%\Image\i7\Set0"
SET _LOG=/LOG:F:\BackupSet0.txt
SET _ACT=/PURGE /R:0 /W:0 /NDL /NFL %_LOG%
START "" /WAIT Robocopy %_Set0% K:\Images_Set0 %_ACT%

I suggest monitoring the file size being written to view what is
happening. This can be done using a batch loop with SET _Size=%%~zI

Where did you get Robocopy 6.1.7601.23403?



"C:\Windows\System32\Robocopy.exe" Properties

Just did an Everything search for RoboCopy.exe "Executable:"

5.1.10.1027
"C:\Windows\winsxs\amd64_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.17514_none_252d 34f00303c6fa\Robocopy.exe"


6.1.7601.23403
"C:\Windows\winsxs\amd64_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.23403_none_25c0 70011c1a6a70\Robocopy.exe"


5.1.10.1027
"C:\Windows\winsxs\x86_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.17514_none_c90e 996c4aa655c4\Robocopy.exe"


6.1.7601.23403
"C:\Windows\winsxs\x86_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.23403_none_c9a1 d47d63bcf93a\Robocopy.exe"


Two are 5.1.10.1027 and two 6.1.7601.23403 **BUT** why does the list
**NOT** include "C:\Windows\System32\Robocopy.exe" ???

Nothing in the RoboCopy Log indicates which version was being used but
since CMD comes up using C:\Windows\System32 I expect
"C:\Windows\System32\Robocopy.exe".

C:\Windows\System32\Robocopy.exe is a hardlinked object.

A filenum (entry in the $MFT) can have multiple file pointers.
Each one appears in the file system as if it was a separate file,
but they're all handled under the sane filenum entry.

The "nfi.exe" utility can show when a file has two or
more $FILE_NAME entries. But the utility won't print out the
contents of the hard links.

File 25918
\Windows\System32\Robocopy.exe
$STANDARD_INFORMATION (resident)
$FILE_NAME (resident) --- the WinSXS one ???
$FILE_NAME (resident) --- the System32 one ???
$DATA (nonresident)
logical sectors 20151712-20151967 (0x1337da0-0x1337e9f)

It turns out that Everything.exe just reads the first one, and
not any of the others.

https://www.voidtools.com/forum/viewtopic.php?t=459

"Only the first hardlink of a file will be indexed and monitored.
Files that are not the first hardlink will not be indexed or monitored.

This is a limitation of the USN Change Journal.

I have plans to index all hard links in the future.
However, you will have to update the indexes manually
as the USN Change Journal does not support hardlinks."

Now, if you run the nfi.exe command (a utility from Win2K days),
it lists all the winsxs entries first. Because, naturally, first
a file is stored in winsxs (as part of Component Based Servicing
or CBS), and afterwards the file is hardlinked into C:\Windows\System32.

So that gives a kind of answer, as to why you can see all
the WinSXS entries, but the actual entry that counts, the
one we use, because it comes second in the $MFT table used
to store the pointer, Everything.exe cannot list it.

*******

Something else I just learned, is more disturbing. I saw a
thread very recently, where someone was claiming the release
control system was busted on Robocopy. Now, I didn't pay this
any mind, because I was concentrating on something else.

However, now I realize the gravity of what that person was saying.

Try this:

In an Administrator Command Prompt window...

fsutil hardlink list C:\Windows\System32\robocopy.exe

Mine comes back with this.

\Windows\System32\Robocopy.exe

\Windows\winsxs\amd64_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.17514_none_252d 34f00303c6fa\Robocopy.exe

The fricken *folder* in WinSXS has a release number
with the correct format for Win7SP1. A value of 7600
is the original Windows. And 7601 is SP1. "6.1.7601.17514"

However, if you do Properties on the file,
*all* the files use the (dishonest) 5.1.10.1027 identifier.

So for this file, the *only* way to get the release
information, is using the fsutil command!!! All the
files being hardlinked into the system32 folder,
will have the same 5.1.10.1027 number each time.

So if you were to complain to Microsoft

"My 5.1.10.1027 robocopy.exe is broken..."

the staff at Microsoft would laugh at you. Because
the actual number (6.1.7601.17514) is only available
by an indirect fsutil method.

So even if David had followed my instructions to
install the Convenience Rollup, visually, doing Properties
on the file would show 5.1.10.1027, and he'd tell
me I was full of ****.

*******

So *now* I understand, why what looked like a crusty copy
of Robocopy on my system, wasn't actually that crusty.
And we won't know until David runs the fsutil command,
what he is actually using.

How does this stuff happen ?

Simple.

Robocopy started life as an article on a webpage called
"Utility Spotlight". That means, it was the private
creation of a Microsoft employee and not part of the OS.

Later, Microsoft decided to bundle the file with the OS.

However, this release number snafu, indicates the stubborn
people at Microsoft, *will not* support this program
as if it is part of the OS. By *not* stamping robocopy,exe
properties with "6.1.7601.17514", they're saying it is
not part of the OS. And that's why the release number
involved here is "broken". That's a subtle hint - if you
ask for help, they will give you the middle finger.

Purely a guess,
Paul

On 1/21/2017 12:57 PM, Paul wrote:
Zaidy036 wrote:
On 1/21/2017 10:46 AM, Zaidy036 wrote:
On 1/21/2017 12:55 AM, David E. Ross wrote:
On 1/20/2017 9:09 AM, Zaidy036 wrote:
On 1/20/2017 10:04 AM, Mr. Man-wai Chang wrote:
On 12/1/2017 2:37 PM, David E. Ross wrote:
For files of size 1.5 GB or less, Robocopy.exe terminates, allowing
the
script to continue with additional commands. For files of size 10
GB or
more, Robocopy.exe does NOT terminate, preventing the script to
continue. The same script either has the problem (file larger
than 10
GB) or does not have the problem (file less than 1.5 GB).

I am not sure where between 1.5 GB and 10 GB the problem arises.

Robocopy is a 32-bit program.

I think the maximum size of file should be limited to 2 GB?

Need to port and compile the program using 64-bit C/C++ compilers?

For the meantime, use a 64-bit 7-zip to compress and split the 10GB
files into multiple 1.5GB files?


Win 7 Home 64 bit

Robocopy 6.1.7601.23403

I have a batch using Robocopy from NAS over CAT 5 to PC to USB3 1
TB HDD
to backup an Acronis Images files (tib) 500GB (full) and 30 to 50 GB
(incremental) without problems except the time it takes to finish
writing the larger file.

Here are my central batch cmds:
SET "ipNAS=\\xxx.xxx.x.xx"
SET "_Set0=%ipNAS%\Image\i7\Set0"
SET _LOG=/LOG:F:\BackupSet0.txt
SET _ACT=/PURGE /R:0 /W:0 /NDL /NFL %_LOG%
START "" /WAIT Robocopy %_Set0% K:\Images_Set0 %_ACT%

I suggest monitoring the file size being written to view what is
happening. This can be done using a batch loop with SET _Size=%%~zI

Where did you get Robocopy 6.1.7601.23403?



"C:\Windows\System32\Robocopy.exe" Properties

Just did an Everything search for RoboCopy.exe "Executable:"

5.1.10.1027
"C:\Windows\winsxs\amd64_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.17514_none_252d 34f00303c6fa\Robocopy.exe"


6.1.7601.23403
"C:\Windows\winsxs\amd64_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.23403_none_25c0 70011c1a6a70\Robocopy.exe"


5.1.10.1027
"C:\Windows\winsxs\x86_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.17514_none_c90e 996c4aa655c4\Robocopy.exe"


6.1.7601.23403
"C:\Windows\winsxs\x86_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.23403_none_c9a1 d47d63bcf93a\Robocopy.exe"


Two are 5.1.10.1027 and two 6.1.7601.23403 **BUT** why does the list
**NOT** include "C:\Windows\System32\Robocopy.exe" ???

Nothing in the RoboCopy Log indicates which version was being used but
since CMD comes up using C:\Windows\System32 I expect
"C:\Windows\System32\Robocopy.exe".

C:\Windows\System32\Robocopy.exe is a hardlinked object.

A filenum (entry in the $MFT) can have multiple file pointers.
Each one appears in the file system as if it was a separate file,
but they're all handled under the sane filenum entry.

The "nfi.exe" utility can show when a file has two or
more $FILE_NAME entries. But the utility won't print out the
contents of the hard links.

File 25918
\Windows\System32\Robocopy.exe
$STANDARD_INFORMATION (resident)
$FILE_NAME (resident) --- the WinSXS one ???
$FILE_NAME (resident) --- the System32 one ???
$DATA (nonresident)
logical sectors 20151712-20151967 (0x1337da0-0x1337e9f)

It turns out that Everything.exe just reads the first one, and
not any of the others.

https://www.voidtools.com/forum/viewtopic.php?t=459

"Only the first hardlink of a file will be indexed and monitored.
Files that are not the first hardlink will not be indexed or monitored.

This is a limitation of the USN Change Journal.

I have plans to index all hard links in the future.
However, you will have to update the indexes manually
as the USN Change Journal does not support hardlinks."

Now, if you run the nfi.exe command (a utility from Win2K days),
it lists all the winsxs entries first. Because, naturally, first
a file is stored in winsxs (as part of Component Based Servicing
or CBS), and afterwards the file is hardlinked into C:\Windows\System32.

So that gives a kind of answer, as to why you can see all
the WinSXS entries, but the actual entry that counts, the
one we use, because it comes second in the $MFT table used
to store the pointer, Everything.exe cannot list it.

*******

Something else I just learned, is more disturbing. I saw a
thread very recently, where someone was claiming the release
control system was busted on Robocopy. Now, I didn't pay this
any mind, because I was concentrating on something else.

However, now I realize the gravity of what that person was saying.

Try this:

In an Administrator Command Prompt window...

fsutil hardlink list C:\Windows\System32\robocopy.exe

Mine comes back with this.

\Windows\System32\Robocopy.exe


\Windows\winsxs\amd64_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.17514_none_252d 34f00303c6fa\Robocopy.exe


The fricken *folder* in WinSXS has a release number
with the correct format for Win7SP1. A value of 7600
is the original Windows. And 7601 is SP1. "6.1.7601.17514"

However, if you do Properties on the file,
*all* the files use the (dishonest) 5.1.10.1027 identifier.

So for this file, the *only* way to get the release
information, is using the fsutil command!!! All the
files being hardlinked into the system32 folder,
will have the same 5.1.10.1027 number each time.

So if you were to complain to Microsoft

"My 5.1.10.1027 robocopy.exe is broken..."

the staff at Microsoft would laugh at you. Because
the actual number (6.1.7601.17514) is only available
by an indirect fsutil method.

So even if David had followed my instructions to
install the Convenience Rollup, visually, doing Properties
on the file would show 5.1.10.1027, and he'd tell
me I was full of ****.

*******

So *now* I understand, why what looked like a crusty copy
of Robocopy on my system, wasn't actually that crusty.
And we won't know until David runs the fsutil command,
what he is actually using.

How does this stuff happen ?

Simple.

Robocopy started life as an article on a webpage called
"Utility Spotlight". That means, it was the private
creation of a Microsoft employee and not part of the OS.

Later, Microsoft decided to bundle the file with the OS.

However, this release number snafu, indicates the stubborn
people at Microsoft, *will not* support this program
as if it is part of the OS. By *not* stamping robocopy,exe
properties with "6.1.7601.17514", they're saying it is
not part of the OS. And that's why the release number
involved here is "broken". That's a subtle hint - if you
ask for help, they will give you the middle finger.

Purely a guess,
Paul
Interesting

fsutil hardlink list C:\Windows\System32\robocopy.exe returned the
following:

\Windows\winsxs\amd64_microsoft-windows-robocopy_31bf3856ad364e35_6.1.7601.23403_none_25c0 70011c1a6a70\Robocopy.exe
\Windows\System32\Robocopy.exe

Note: 6.1.7601.23403

Today, I discovered I merely had to wait a while for Robocopy to
terminate. When I indicate "wait", I mean "wait, wait, and wait".

If copying 1 GB takes 11 seconds (which is indeed the time reported in
the Robocopy log), I would expect copying 10 GB to take ten times
longer: 110 seconds (1.8 minutes). According to the log, however, 10 GB
took over 6 minutes, more than 30 times longer. Nevertheless, the
source file and its copy now have the same SHA512 hash.

The question mark after "SOLVED" is because I am not sure there is a bug
that causes Robocopy to take disproportionally longer to copy
exceptionally large files.

On 1/10/2017 4:35 PM, David E. Ross wrote:
Windows 7 Ultimate SP1 (x64)

When I do my backups, I send the resulting files to an internal drive of
my PC. I then encrypt the files and move them to a removable external
drive that I store remotely.

To move the encrypted files, I use a script that uses Robocopy.exe. A
search tool shows 6 files in my multi-drive system named Robocopy.exe.
The one I am using is located at [C:\Windows\System32\Robocopy.exe],
which has version #5.1.10.1027.

For encrypted files whose sizes are less than 1.5 GB, I have no problem
running the script. For encrypted files whose sizes are in the range 10
GB - 21 GB, however, Robocopy.exe fails to terminate. This means my
script does not complete unless I manually terminate Robocopy.exe via
Windows Task Manager, which then allows my script to continue.

Is there a more recent x64 version of Robocopy than 5.1.10.1027? Is
there something else I could do in my script to facilitate the
termination of Robocopy.exe, possibly in one of its options?



--
David E. Ross
http://www.rossde.com/

When the President of the United States makes a statement of
national importance, I want to see his face as he is talking.
At the least, I want to hear his voice.

Donald: Stop tweeting. Otherwise, how do we know the message
really comes from you?

David E. Ross wrote:
Today, I discovered I merely had to wait a while for Robocopy to
terminate. When I indicate "wait", I mean "wait, wait, and wait".

If copying 1 GB takes 11 seconds (which is indeed the time reported in
the Robocopy log), I would expect copying 10 GB to take ten times
longer: 110 seconds (1.8 minutes). According to the log, however, 10 GB
took over 6 minutes, more than 30 times longer. Nevertheless, the
source file and its copy now have the same SHA512 hash.

The question mark after "SOLVED" is because I am not sure there is a bug
that causes Robocopy to take disproportionally longer to copy
exceptionally large files.


Keep an eye out for System Write Cache activity.

https://s30.postimg.org/69o6so535/sy...rite_cache.gif

Your 10GB transfer, is showing more of the "real" transfer rate.
Not an artificially inflated filling of the system write cache.

The reason the write cache is so exaggerated on that system,
is it has a lot of RAM, so the write cache can be huge. There
seems to be a "percentage" of RAM the system is willing to
allocate (max), before the write cache stops growing. Hitting
the write cache limit is not shown in that picture, because
I had to worry about the whole transaction staying within the
Task Manager window.

I have observed the System Write Cache on both my Win7 and
my Win10 boot systems. WinXP would probably not do that, with
quite as much "style". WinXP was closer to a power-safe system.

The above demo is not power-safe. After the copy dialog disappears
from the screen, if the power goes off within the next 50 seconds,
the next time you boot, the copied file will be "completely missing".
Journal playback will notice the file was not committed, so
its existence will not be acknowledged. No, I'm not going to
turn the power off now, to prove it :-)

Have fun,
Paul