If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
|
Thread Tools | Rate Thread | Display Modes |
#16
|
|||
|
|||
Large file copy behavior question
mick wrote:
On 23/04/2018 14:50:17, Paul wrote: Wolf K wrote: On 2018-04-23 07:06, Paul wrote: Closing Explorer windows (even when it doesn't seem possible the windows in question have a "handle" on the disk), seems to work here. Explorer checks all drives on boot, including external ones. The tree view shows all powered drives. So closing the Explorer window(s) and using the command-line should make for a faster Copy. Or???? If you use Safely Remove, to remove a USB device, sometimes you get a complaint the "volume is busy". The obvious answer, is some program running on your machine, has a file open on the volume in question, perhaps about to write it. Now, if you use Process Explorer or Handle.exe, you can check to see exactly which files are open. And then close the programs that have the files in question open. OK, what happens if Safely Remove *still* won't remove the drive ? It's still busy it seems. One thing you can do, is close the File Explorer windows, as sometimes, even though the File Explorer window isn't displaying the volume contents in question, it still seems to have a handle open on the drive. Even after you've done that, there can still be cases where it won't release so you can unplug it. One source of contention, is the NTFS TxF feature, is keeping TxF files open on the volume. Using the Disk Management "Offline" command seems to help in that case. Shutting down the machine should always release the volume, but you might see a slow shutdown process, if the "culprit" is gasping for breath at shutdown :-) There are some things which don't go gracefully. I had one of these the other day on this machine, and I had to power cycle to regain control. I restored from backup, because I love to do stuff like that... Just to make sure whatever happened, didn't leave any after-effects. The volume wasn't actually damaged, but since I had the backup, I used it. Paul I do not use File Explorer and nothing is using the external disk AFAIK. I have watched the external disk light for ages which is steady, so nothing is writing to it. The external disk is only used for syncing to, using Allway Sync, the only program to be opened and closed in the session. Sometimes the drive can be ejected other times it is held onto saying it is in use. You could try Sysinternals Handle.exe or Process Explorer I think also has the functionality of Handle in it. And see what files are open on that volume. Paul |
Ads |
#17
|
|||
|
|||
Large file copy behavior question
On 23/04/2018 17:57:43, Paul wrote:
mick wrote: On 23/04/2018 14:50:17, Paul wrote: Wolf K wrote: On 2018-04-23 07:06, Paul wrote: Closing Explorer windows (even when it doesn't seem possible the windows in question have a "handle" on the disk), seems to work here. Explorer checks all drives on boot, including external ones. The tree view shows all powered drives. So closing the Explorer window(s) and using the command-line should make for a faster Copy. Or???? If you use Safely Remove, to remove a USB device, sometimes you get a complaint the "volume is busy". The obvious answer, is some program running on your machine, has a file open on the volume in question, perhaps about to write it. Now, if you use Process Explorer or Handle.exe, you can check to see exactly which files are open. And then close the programs that have the files in question open. OK, what happens if Safely Remove *still* won't remove the drive ? It's still busy it seems. One thing you can do, is close the File Explorer windows, as sometimes, even though the File Explorer window isn't displaying the volume contents in question, it still seems to have a handle open on the drive. Even after you've done that, there can still be cases where it won't release so you can unplug it. One source of contention, is the NTFS TxF feature, is keeping TxF files open on the volume. Using the Disk Management "Offline" command seems to help in that case. Shutting down the machine should always release the volume, but you might see a slow shutdown process, if the "culprit" is gasping for breath at shutdown :-) There are some things which don't go gracefully. I had one of these the other day on this machine, and I had to power cycle to regain control. I restored from backup, because I love to do stuff like that... Just to make sure whatever happened, didn't leave any after-effects. The volume wasn't actually damaged, but since I had the backup, I used it. Paul I do not use File Explorer and nothing is using the external disk AFAIK. I have watched the external disk light for ages which is steady, so nothing is writing to it. The external disk is only used for syncing to, using Allway Sync, the only program to be opened and closed in the session. Sometimes the drive can be ejected other times it is held onto saying it is in use. You could try Sysinternals Handle.exe or Process Explorer I think also has the functionality of Handle in it. And see what files are open on that volume. Paul Thanks, I'll have a look at those. -- mick |
#18
|
|||
|
|||
Large file copy behavior question
Andy Burns wrote:
Paul wrote: Before the destination device can be unplugged, the FIFO queue has to be empty. If you were to use Safely Remove, it will need to drain the FIFO queue first. I'm getting more and more frustrated with "safely remove" it mostly seems to claim the drive is still in use, this happens with USB mass storage, USB attached SCSI (UASP) and eSATA drives Various things sometime help, but none seems to help in all cases. Closing all explorer windows Killing all explorer processes Closing all apps and logging out Looking for open handles with procmon It's as if they *want* us to just yank the cable ... grr! I saw that in the meantime, you already have a solution (flip the isOffline and isReadOnly bits on your backup drive), so these are just additional suggestions: - Hibernate the system. Quicker and less invasive than a Restart, etc.. - If your disk is a WD (Western Digital) one, have a look for a WD program, probably in the Notifications Area of the Taskbar. In my case it's 'WD Access' aka 'WD App Manager'. I haven't bothered (yet) looking what it is, how I got it, how to get rid of it, etc., I just Quit it. HTH. |
#19
|
|||
|
|||
Large file copy behavior question
|
#20
|
|||
|
|||
Large file copy behavior question
|
#21
|
|||
|
|||
Large file copy behavior question
In article , Zaidy036
@air.isp.spam says... Make a batch so your work time is not impacted by copy time: I do that. Everything happens while I'm sleeping :-) |
#23
|
|||
|
|||
Large file copy behavior question
btw, I am using plain ol' right-click Copy/Paste for this. There is a more elaborate copy utility in Windows but I can't remember its name just now. I will remember eventually and give that a try. RoboCopy -- Zaidy036 |
#24
|
|||
|
|||
Large file copy behavior question
Jason wrote:
In article , lid says... Isn't Windows marvelous ? First, you have to look at your ecosystem. What is the transfer rate of the source drive ? What is the transfer rate of the destination drive ? Is the destination drive or device, actually USB2, or is a USB3 port and USB3 enclosure controller being used ? It's a garden variety WD USB3 drive connected to a USB3 port. The port is -not- setup for quick removal. I'll give that a try if the culprit may be the (broken?) buffering scheme. btw, I am using plain ol' right-click Copy/Paste for this. There is a more elaborate copy utility in Windows but I can't remember its name just now. I will remember eventually and give that a try. Other times I've delved into Windows' file behavior I think I learned that it does r/w in 64 kB chunks. I don't know if that is configurable. The cluster size on a partition is configurable. The OS partition should be 4K (NTFS). There was even a release or two of Windows 10 that would tolerate other choices on the OS partition, but something happened to that tolerance, and now you should stick with 4K. A data partition can be 64K (and when doing so for NTFS, allows moving the drive to other Windows OSes that understand NTFS). On Windows 10, a larger cluster size than that is allowed. I couldn't believe it when I saw some new options on Win10, but my joy was short-lived when WinXP didn't tolerate the oversized clusters. The CRT (C language runtime) has buffered I/O, where the maximum buffer size supported seems to be related to the size of a cluster. I've written a couple of small utilities where I use that, and it makes the runtime responsible for buffering, rather than my program. You can get at least 300MB/sec out of that. You can always write your own buffering routine, to do transactions in a larger size than that. Robocopy uses the non-blocking I/O features the system has to offer, and you can have reads and writes overlapped during a copy. But the egregious behaviors of Windows 10, swamp out any nuances, so really there's no reason to try to optimize anything. The OS decides whether you're late for an appointment - you don't. It's like they don't have any staff in the storage department, watching the shop. If I go back to a pre-Vista OS, I'm more likely to see consistency, enough to make me tune a few things. (Use the download button at the top of the page, to get the original-sized image. The image as submitted is 768x998.) https://postimg.cc/image/6wofuqb07/ To put that in perspective, the original release candidate for Win10 had a flat-as-a-pancake transfer curve for that test case. Too bad the transfer rate in that release, was only *1GB/sec*. In other words, consistent... but slow. Now, nobody really needs consistency. You're not supposed to be looking at screens like that - "just concentrate on updating your Facebook please, stop looking at perfmon". Paul |
#25
|
|||
|
|||
Large file copy behavior question
Jason wrote:
In article , Zaidy036 @air.isp.spam says... Make a batch so your work time is not impacted by copy time: I do that. Everything happens while I'm sleeping :-) I hope you're running that software that delays reboots while you're away :-) (Maybe you've delayed updates or something, and that's why it's not an issue.) No, I'm not talking about whacked-out pages like this (IT treadmill). https://docs.microsoft.com/en-us/win...e/waas-restart This thing appears to use an AutoHotKey script, packaged as an EXE. If you run the EXE, the idea is, if a Windows Update comes in while you're away, it "moves the goalposts" on the Active Hours thing. https://github.com/cryptogeek/NoReboot Paul |
#26
|
|||
|
|||
Large file copy behavior question
Jason wrote:
I (thought) I corrected the units... Yeah you did in your follow-up |
#27
|
|||
|
|||
Large file copy behavior question
|
|
Thread Tools | |
Display Modes | Rate This Thread |
|
|