Speeds of Data Copying in W7

I have an external hard drive connected to USB2 port on 64-bit W7 Home
Premium. I use it to make backup copies of my data using windows
explorer copy/paste.

My data is in a separate folder called coincidentally My_Data.

I initiated a copy/paste from My_Data on the W7 drive to a same-name
folder on the external drive. I see the transfer is said to be
8MB/sec. Then I initiated a second copy/paste of my three 'desktop',
'downloads' and 'documents' folders on the W7 drive to same-named
folders on the same external drive. The transfer is said to be
19MB/sec.

Why the different speeds? I certainly would prefer the faster speed.
As it is, the slower transfer will take more than a day to complete.
Not good.

Thanks

ME

On Sat, 24 Sep 2016 08:34:43 -0400, wrote:

I initiated a copy/paste from My_Data on the W7 drive to a same-
name folder on the external drive. I see the transfer is said to be
8MB/sec. Then I initiated a second copy/paste of my three
'desktop', 'downloads' and 'documents' folders on the W7 drive to
same-named folders on the same external drive. The transfer is said
to be 19MB/sec.

Why the different speeds?

I can't answer your question, but if you really mean Copy/Paste,
you're putting a huge amount of data on the clipboard. As an
alternative, consider Robocopy, which is blindingly fast -- it's
already on your computer as part of Windows.

Of course you're limited by USB2 speeds, but when I copy to my USB2
external hard drive, just by dragging files from one Explorer window
to another, I get 20+ MB/s. So I wonder if going through the
clipboard is what is slowing your speeds by so much.

Bare-bones robocopy documentation:
https://technet.microsoft.com/en-us/...(v=ws.11).aspx

Intro to Robocopy:
http://www.thewindowsclub.com/whats-...ows-7-robocopy

There's also a Microsoft GUI, which I haven't tried:
http://download.microsoft.com/downlo...8a1-4f71-8546-
25c359cc0842/UtilitySpotlight2006_11.exe

--
Stan Brown, Oak Road Systems, Tompkins County, New York, USA
http://BrownMath.com/
http://OakRoadSystems.com/
Shikata ga nai...

In message , Stan Brown
writes:
On Sat, 24 Sep 2016 08:34:43 -0400, wrote:

I initiated a copy/paste from My_Data on the W7 drive to a same-
[]
I can't answer your question, but if you really mean Copy/Paste,
you're putting a huge amount of data on the clipboard. As an
[]
Does it really? I've often got the impression that cut, at least, from
Windows Explorer _doesn't_ actually cut until you do the paste (at least
if it's between drives), and I think similarly with copy. I _think_ the
only things that get cut/copied when you do Ctrl-C (or select Copy from
a menu) are the filenames (and pointer data).

I guess it'd be fairly easy to check: initiate a cut or copy (without
doing a paste) of the largest file you can find; if there is (much!)
disc activity _at the point you select/initiate copy_, then I'm wrong.

Of course, it _could_ still go via some notional clipboard when you
finally do the paste, but I don't think so, I think it copies the files
direct.
--
J. P. Gilliver. UMRA: 1960/1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

War is God's way of teaching Americans geography. -Ambrose Bierce, writer
(1842-1914)

wrote:
I have an external hard drive connected to USB2 port on 64-bit W7 Home
Premium. I use it to make backup copies of my data using windows
explorer copy/paste.

My data is in a separate folder called coincidentally My_Data.

I initiated a copy/paste from My_Data on the W7 drive to a same-name
folder on the external drive. I see the transfer is said to be
8MB/sec. Then I initiated a second copy/paste of my three 'desktop',
'downloads' and 'documents' folders on the W7 drive to same-named
folders on the same external drive. The transfer is said to be
19MB/sec.

Why the different speeds? I certainly would prefer the faster speed.
As it is, the slower transfer will take more than a day to complete.
Not good.

Thanks

ME


It's normal. I have USB 3.0 and one large file will go at 300 MB a sec
but a large bunch of files will only do 40 to 50.

--
Z

wrote:
I have an external hard drive connected to USB2 port on 64-bit W7 Home
Premium. I use it to make backup copies of my data using windows
explorer copy/paste.

My data is in a separate folder called coincidentally My_Data.

I initiated a copy/paste from My_Data on the W7 drive to a same-name
folder on the external drive. I see the transfer is said to be
8MB/sec. Then I initiated a second copy/paste of my three 'desktop',
'downloads' and 'documents' folders on the W7 drive to same-named
folders on the same external drive. The transfer is said to be
19MB/sec.

Why the different speeds? I certainly would prefer the faster speed.
As it is, the slower transfer will take more than a day to complete.
Not good.

Thanks

ME

The seek time of any hard drive involved in the transfer
is not zero. Imagine 1KB files which are spread all over
the platter surfaces. If it takes 12 milliseconds for the heads to
move to the other part of the disk, you can only transfer
80 files per second. If each file is 1KB in size, your
transfer rate is 80KB per second. It's not unusual to
see 1MB/sec transfer rates, when the files are small and
scattered.

If you reduce the number of seeks, and increase the
size of each individual file (to encourage sequential
reads or writes on the hard drive), then the performance
should pick up. If you were transferring four DVD-sized
ISO files to the external drive, that would happen
at 30MB/sec. Because there might only be four large
head movements, while the rest of the time, the
transfer was purely sequential.

If all the files in question, were stored in a single 7ZIP
file (7-zip.org), then the ZIP file would copy over at
30MB/sec. But of course, the mere act of making a ZIP
of the scattered files, also invokes the 12 millisecond
hard drive seek time, so as a practical matter, the
idea has no merit. It would take far longer to make the
ZIP than time saved during the transfer.

Note that, even if both the source drive and the destination
drive were SSD based, the transfer would still be a little
bit slow. The file system has an upper limit on how many
files it can process per second (maybe 5000 to 10000 files,
when the IOP rate of the SSD could be 100,000 to 200,000).
And this is why technologies like NVMe are kinda pointless
at times, because the file system performance has not scaled
at the same rate as the Flash storage is speeding up.

*******

You can get USB3 cards for both PCI slot and PCI Express slot.
Because the PCI version costs twice as much, and the designs
aren't kept on the market due to poor sales, there's no guarantee
on a given day that you can find one for sale.

PCI USB3 - 100MB/sec (bus limited, board uses bridge chip and USB3 chip)
PCI Express Rev.1.1 USB3 - approx. 200MB/sec, bus limited.
PCI Express Rev.2 USB3 - approx. 400MB/sec, bus limited.
Southbridge USB3 - maybe 450 to 500MB/sec via UASP protocol

The best USB3 is the port on a Southbridge, and this is due
to the roughly x4 lanes of transfer bandwidth in the chipset,
rather than some "design genius" in the block itself. It's
because the PCI Express versions don't have a wide enough
bus interface.

So even if you have an older computer, with only PCI
slots, you can still get better speed. You would need
a new enclosure for the external disk, with its own USB3
peripheral chip, to get that speed. I got a couple USB3
enclosures here for around $35, but they top out at around
200MB/sec. Which means, "they're good enough" :-)
When a 200MB/sec enclosure is plugged into a 500MB/sec port,
you only get 200MB/sec, and only on sequential transfer.
If I repeat your "small file" exercise, I can expect 1MB/sec
even with the faster port.

So it *still* doesn't solve the small file problem. But it
does help when you want to do a full backup of C: .

*******

When Macrium Reflect Free or other VSS based backup programs
make backups, they use sequential transfer techniques. So the
head *doesn't* fly all over the disk during the backup. And
the rate can be much higher as a result. The software makes
a map of what clusters need to be transferred first, so it
can plan the least disruptive head movements on the disk.
In some rare circumstances, it may be faster to transfer the
whole partition, simply because of the reduction in head
movement.

But if your rate is 8MB/sec and the best case is 30MB/sec
on the destination device, there isn't enough elbow room there
to make it worthwhile fiddling with the transfer. Just do it,
and wait the whole day or whatever. Many times I try novel
transfer methods, only to realize later that I wasted more
time setting up the novel method, than I could ever save in
practice. If I had a 100-200MB/sec pathway to destination
storage, then I might use Macrium and just back up the
entire partition. It depends on how "miserably small" the
materials are that need to be backed up, whether I
would resort to such a technique.

Paul

On Sat, 24 Sep 2016 15:52:19 +0100, J. P. Gilliver (John) wrote:
In message , Stan Brown
writes:
On Sat, 24 Sep 2016 08:34:43 -0400, wrote:

I initiated a copy/paste from My_Data on the W7 drive to a same-
[]
I can't answer your question, but if you really mean Copy/Paste,
you're putting a huge amount of data on the clipboard. As an
[]
Does it really? I've often got the impression that cut, at least, from
Windows Explorer _doesn't_ actually cut until you do the paste (at least
if it's between drives), and I think similarly with copy. I _think_ the
only things that get cut/copied when you do Ctrl-C (or select Copy from
a menu) are the filenames (and pointer data).

What you say sounds reasonable, as I think about it. I think most
likely you're right and I was wrong.

Others have already given a believable explanation for the speed
difference: the penalty from doing seeks for lots of files.

--
Stan Brown, Oak Road Systems, Tompkins County, New York, USA
http://BrownMath.com/
http://OakRoadSystems.com/
Shikata ga nai...

On Sat, 24 Sep 2016 08:57:14 -0400, Stan Brown
wrote:

On Sat, 24 Sep 2016 08:34:43 -0400, wrote:

I initiated a copy/paste from My_Data on the W7 drive to a same-
name folder on the external drive. I see the transfer is said to be
8MB/sec. Then I initiated a second copy/paste of my three
'desktop', 'downloads' and 'documents' folders on the W7 drive to
same-named folders on the same external drive. The transfer is said
to be 19MB/sec.

Why the different speeds?

I can't answer your question, but if you really mean Copy/Paste,
you're putting a huge amount of data on the clipboard. As an
alternative, consider Robocopy, which is blindingly fast -- it's
already on your computer as part of Windows.

Of course you're limited by USB2 speeds, but when I copy to my USB2
external hard drive, just by dragging files from one Explorer window
to another, I get 20+ MB/s. So I wonder if going through the
clipboard is what is slowing your speeds by so much.

Bare-bones robocopy documentation:
https://technet.microsoft.com/en-us/...(v=ws.11).aspx

Intro to Robocopy:
http://www.thewindowsclub.com/whats-...ows-7-robocopy

There's also a Microsoft GUI, which I haven't tried:
http://download.microsoft.com/downlo...8a1-4f71-8546-
25c359cc0842/UtilitySpotlight2006_11.exe
Robocopy worked fine for me in my W7. A little tricky, but a lot
faster.

Thanks
me

On Sun, 25 Sep 2016 05:03:06 -0400, wrote:

Robocopy worked fine for me in my W7. A little tricky, but a lot
faster.

Glad you like it -- I'm a big fan.

It was recommended to me here a year or so ago, and I use it to keep
my home computers. It's a little tricky, as you say, but I dedicated
a couple of hours to reading the documentation carefully and then
stored the options I always want to use, in an alias "robu".

BTW, there's a user manual, which I didn't realize I had a link for:
http://theether.net/download/Microso...s/robocopy.pdf

--
Stan Brown, Oak Road Systems, Tompkins County, New York, USA
http://BrownMath.com/
http://OakRoadSystems.com/
Shikata ga nai...

On Sat, 24 Sep 2016 15:52:19 +0100, "J. P. Gilliver (John)"
wrote:

In message , Stan Brown
writes:
On Sat, 24 Sep 2016 08:34:43 -0400, wrote:

I initiated a copy/paste from My_Data on the W7 drive to a same-
[]
I can't answer your question, but if you really mean Copy/Paste,
you're putting a huge amount of data on the clipboard. As an
[]
Does it really? I've often got the impression that cut, at least, from
Windows Explorer _doesn't_ actually cut until you do the paste (at least
if it's between drives), and I think similarly with copy. I _think_ the
only things that get cut/copied when you do Ctrl-C (or select Copy from
a menu) are the filenames (and pointer data).
Actually, it seems it acts like the all of the upper level filenames
are stored in a buffer someplace.

Thus, if you select a small group directory entries then the selection
operation will be pretty fast, but if you have 1000 files or folders
selected then the operation will take a while. Contrast this
operation
with a command window COPY, where the names looked at only as the
COPY operation gets to it.

So, an "Explorer" selection takes linear in the number of
files/folders/links/etc selected initially. The execution of the
"Copy" or {"Cut" that requires a copy since it is to another device}
and should be expected to run in linear time with
the total amount of data that needs to be copied, with additional
time linear in the total number of entries needing to be copied.

I have not attempted to analyze why the speeds for copies of
a single large file varies between all of the various methods
that I use. I have noticed that some programs attempt to read large
amounts of data then write large amounts of data, which obviously
speeds things up on spinning disks and which also speeds things
up considerably on SSDs and even more on many USB keys.
(This is contrasted to using the "rookie" method of just
using multiple large buffers, letting the "disk" order the
operations, and sharing the buffers to avoid in memory moves.)

I guess it'd be fairly easy to check: initiate a cut or copy (without
doing a paste) of the largest file you can find; if there is (much!)
disc activity _at the point you select/initiate copy_, then I'm wrong.
Yes, that is what I confirmed after noticing the timing.

Of course, it _could_ still go via some notional clipboard when you
finally do the paste, but I don't think so, I think it copies the files
direct.

On Sun, 25 Sep 2016 10:06:44 -0400, Stan Brown
wrote:

On Sun, 25 Sep 2016 05:03:06 -0400, wrote:

Robocopy worked fine for me in my W7. A little tricky, but a lot
faster.

Glad you like it -- I'm a big fan.

It was recommended to me here a year or so ago, and I use it to keep
my home computers. It's a little tricky, as you say, but I dedicated
a couple of hours to reading the documentation carefully and then
stored the options I always want to use, in an alias "robu".

BTW, there's a user manual, which I didn't realize I had a link for:
http://theether.net/download/Microso...s/robocopy.pdf


Thanks

Me

On 24/09/2016 13:34, wrote:
I certainly would prefer the faster speed.
As it is, the slower transfer will take more than a day to complete.
Not good.

Thanks


Use the command prompt and type something like this:

xcopy *.* /s /v /e f:\

This assumes you are in your My_Data folder and your target drive is F:\

Now go and make yourself a cup of tea and also go for quick pee and then
come back to see the progress.




--
With over 350 million devices now running Windows 10, customer
satisfaction is higher than any previous version of windows.

On Mon, 26 Sep 2016 01:43:11 +0100, Good Guy
wrote:

On 24/09/2016 13:34, wrote:
I certainly would prefer the faster speed.
As it is, the slower transfer will take more than a day to complete.
Not good.

Thanks


Use the command prompt and type something like this:

xcopy *.* /s /v /e f:\

This assumes you are in your My_Data folder and your target drive is F:\

Now go and make yourself a cup of tea and also go for quick pee and then
come back to see the progress.

I will
ME

On Mon, 26 Sep 2016 05:21:34 -0400, wrote:

On Mon, 26 Sep 2016 01:43:11 +0100, Good Guy
wrote:

On 24/09/2016 13:34, wrote:
I certainly would prefer the faster speed.
As it is, the slower transfer will take more than a day to complete.
Not good.

Thanks


Use the command prompt and type something like this:

xcopy *.* /s /v /e f:\

This assumes you are in your My_Data folder and your target drive is F:\

Now go and make yourself a cup of tea and also go for quick pee and then
come back to see the progress.

I will
ME


Damn!

I just re-tried installing the driver and it seems to have worked.
Device Manager is happy, and the task bar shows connection.
Sorry to have bothered you guys

Alan