folermsize limits in Windows 10

what is the maximum folder size in terms of bytes and/or number of
files. I am digitizing very large a larg number of photos and may
have a problem.

Ben Lewis

Bentot wrote:

what is the maximum folder size in terms of bytes and/or number of
files. I am digitizing very large a larg number of photos and may
have a problem.

NTFS, eFAT, or FAT32, FAT/FAT16, or some other file system?

NTFS
Max files on disk: 4,294,967,295
Max files in folder: 4,294,967,295

FAT32
Max files on disk: 268,435,437
Max files in folder: 65,534

FAT[16]
Max files on disk: 65,517
Max files in folder: 512 (128 for root folder)

Extracted from:
http://ask-leo.com/is_there_a_limit_..._can_hold.html

Be sure to DISABLE the thumbview on those folders; else, you'll be
waiting eons for the green progress bar to paint inside the address bar
while snapping a pic to store in the thumbnail cache. Use the List or
Details views and the "Always show icons, never thumbnails" view option.
Just in case Windows decides the folder is a Pictures folder type,
right-click on the folder, go to Customize tab (if exists), and change
the "Optimize this folder for:" option from Pictures to General Items.

On 26.09.19 4:22, Bentot wrote:
what is the maximum folder size in terms of bytes and/or number of
files. I am digitizing very large a larg number of photos and may
have a problem.

Ben Lewis

So zip them, and avoid very deep(and inefficient) dirs.

Bentot wrote:
what is the maximum folder size in terms of bytes and/or number of
files. I am digitizing very large a larg number of photos and may
have a problem.

Ben Lewis

The theoretical limit and the practical limit
are two different things.

File Explorer is a bit of a pig.

You can start to see performance problems, at
the 60,000 file level in a single folder.
If you delete 5000 of the files, File Explorer
may not be able to finish re-drawing the screen.
And it's not out of RAM either. File Explorer
will stop mallocing RAM, at around 15GB or so
on a 64-bit system.

Does Microsoft care ?

I've not seen anything to suggest this bothers them.

Just as the ****ty performance of Notepad didn't
bother them for the longest time. Today in Windows 10,
you can actually use the Find/Replace dialog in Notepad
and actually get something done. And that *is* different.
Notepad still cannot handle a large file. It's address space
might be limited to 2GB of Unicode characters or something,
but the dialog warning about the size is triggered before
that happens.

I did use Notepad++ to edit a 1.5GB file, but I think
Notepad++ cannot handle much bigger than that. Notepad++
is still better than Notepad, but is not "indefinitely
expandable". And tools that cover 64-bit address
spaces, tend to be web based or are "sliding viewer viewers"
rather than editors.

Your computer is filled with limits (and makes a lot of
promises it cannot keep). You'd swear it was 1990 or something,
when you run into these limits.

Paul

In article ,
says...

On Wed, 25 Sep 2019 19:22:14 -0700, Bentot wrote:

what is the maximum folder size in terms of bytes and/or number of
files. I am digitizing very large a larg number of photos and may
have a problem.

Ben Lewis

I don't know what Win 10 holds, but why would it be any less than my
Win XP?

The total Gigs of the sub folders in one of my main video folders is
172 Gigabytes.

One of the sub folders alone contains 75 Gigs of videos.

I have almost 150 folders containing various videos.

I have an ezternal drive with one of several folders contains 8070 sub-
folders as I write, total over 150,000 files in those subfolders.
Another 1000+ files in another separate folder on same drive. (music and
live concerts storage/backup).

Two separate but contents identical (back of backup) externals. No
problems with either.

On Wed, 25 Sep 2019 22:23:56 -0500, VanguardLH wrote:

Bentot wrote:

what is the maximum folder size in terms of bytes and/or number of
files. I am digitizing very large a larg number of photos and may
have a problem.

NTFS, eFAT, or FAT32, FAT/FAT16, or some other file system?

NTFS
Max files on disk: 4,294,967,295
Max files in folder: 4,294,967,295

FAT32
Max files on disk: 268,435,437
Max files in folder: 65,534

FAT[16]
Max files on disk: 65,517
Max files in folder: 512 (128 for root folder)

Extracted from:
http://ask-leo.com/is_there_a_limit_..._can_hold.html

Be sure to DISABLE the thumbview on those folders; else, you'll be
waiting eons for the green progress bar to paint inside the address bar
while snapping a pic to store in the thumbnail cache. Use the List or
Details views and the "Always show icons, never thumbnails" view option.
Just in case Windows decides the folder is a Pictures folder type,
right-click on the folder, go to Customize tab (if exists), and change
the "Optimize this folder for:" option from Pictures to General Items.

Exactly what I was looking for! I am using NTFS.

Now, back to troubleshooting

Thank you Vanguard.

Ben Lewis

wrote:

On Wed, 25 Sep 2019 19:22:14 -0700, Bentot wrote:

what is the maximum folder size in terms of bytes and/or number of
files. I am digitizing very large a larg number of photos and may
have a problem.

Ben Lewis

I don't know what Win 10 holds, but why would it be any less than my
Win XP?

It's a factor of the file system, not the operating system. Both XP
and Win10 can use NTFS, which has effectively no limit on folder size,
as detailed in an earlier post.

--
Tim Slattery
tim at risingdove dot com

Paul wrote:

You can start to see performance problems, at the 60,000 file level in
a single folder. If you delete 5000 of the files, File Explorer may
not be able to finish re-drawing the screen. And it's not out of RAM
either. File Explorer will stop mallocing RAM, at around 15GB or so
on a 64-bit system.

The Recycle Bin is a FIFO cache: when its reserved space gets consumed,
it will delete the oldest deleted files until there is enough reserved
space for the new deleted file. It doesn't look at the total size of
all candidate delete files to first deleted all the oldest files to make
room (to make it one operation to removed deleted files all at once to
make room for all the newly deleted files). It does it on a one-to-one
basis: remove old deleted files until there is room for a new deleted
file, and repeat for each new deleted file in the pending list. This
will take LOTS of time. In fact, if you delete lots of files, some of
those just deleted might end up getting removed to make room for the
latest deleted files. Moving that view port of deleted files in the
Recycle Bin to keep the last N files within the reserve space incurs a
lot of overhead.

You could flush the Recycle Bin and hope the number of files you are
going to delete does not exceed the reserve space for the Recycle Bin.
Because I can't be sure how many of the old deleted files will get
removed, and because I likely don't care, I simply disabled the Recycle
Bin when deleting (moving) many thousands of files. A huge delete
operation (by file count) goes much faster when the Recycle Bin is
disabled.

On Thu, 26 Sep 2019 19:06:25 -0500, VanguardLH wrote:

Paul wrote:

You can start to see performance problems, at the 60,000 file level in
a single folder. If you delete 5000 of the files, File Explorer may
not be able to finish re-drawing the screen. And it's not out of RAM
either. File Explorer will stop mallocing RAM, at around 15GB or so
on a 64-bit system.

The Recycle Bin is a FIFO cache: when its reserved space gets consumed,
it will delete the oldest deleted files until there is enough reserved
space for the new deleted file. It doesn't look at the total size of
all candidate delete files to first deleted all the oldest files to make
room (to make it one operation to removed deleted files all at once to
make room for all the newly deleted files). It does it on a one-to-one
basis: remove old deleted files until there is room for a new deleted
file, and repeat for each new deleted file in the pending list. This
will take LOTS of time. In fact, if you delete lots of files, some of
those just deleted might end up getting removed to make room for the
latest deleted files. Moving that view port of deleted files in the
Recycle Bin to keep the last N files within the reserve space incurs a
lot of overhead.

You could flush the Recycle Bin and hope the number of files you are
going to delete does not exceed the reserve space for the Recycle Bin.
Because I can't be sure how many of the old deleted files will get
removed, and because I likely don't care, I simply disabled the Recycle
Bin when deleting (moving) many thousands of files. A huge delete
operation (by file count) goes much faster when the Recycle Bin is
disabled.

When you say you disabled the Recycle Bin, is it safe to say that you
simply pressed Shift when you initiated the Delete operation, in order to
temporarily bypass the Recycle Bin?

On Thu, 26 Sep 2019 07:41:55 -0700, Bentot wrote:

On Wed, 25 Sep 2019 22:23:56 -0500, VanguardLH wrote:

Bentot wrote:

what is the maximum folder size in terms of bytes and/or number of
files. I am digitizing very large a larg number of photos and may
have a problem.

NTFS, eFAT, or FAT32, FAT/FAT16, or some other file system?

NTFS
Max files on disk: 4,294,967,295
Max files in folder: 4,294,967,295

FAT32
Max files on disk: 268,435,437
Max files in folder: 65,534

FAT[16]
Max files on disk: 65,517
Max files in folder: 512 (128 for root folder)

Extracted from:
http://ask-leo.com/is_there_a_limit_..._can_hold.html

Be sure to DISABLE the thumbview on those folders; else, you'll be
waiting eons for the green progress bar to paint inside the address bar
while snapping a pic to store in the thumbnail cache. Use the List or
Details views and the "Always show icons, never thumbnails" view option.
Just in case Windows decides the folder is a Pictures folder type,
right-click on the folder, go to Customize tab (if exists), and change
the "Optimize this folder for:" option from Pictures to General Items.

Exactly what I was looking for! I am using NTFS.

Now, back to troubleshooting

Thank you Vanguard.

Ben Lewis

I found the problem. I didn't notice that the file nubering had
reached the end 0f that 1000 group,and doubbled back and cause
overwrite a dozen previously scanned photos photo files. The files
were bing saved, but not wgere I expected. A smal foopa. But I can
live with it. The file numbers is made up of three alphebet
charathers followed by thre numnerals. When it got to 999, it didn't
increase the third letter one tick. Had me stupped for a bit.

Char Jackson wrote:

On Thu, 26 Sep 2019 19:06:25 -0500, VanguardLH wrote:

Paul wrote:

You can start to see performance problems, at the 60,000 file level in
a single folder. If you delete 5000 of the files, File Explorer may
not be able to finish re-drawing the screen. And it's not out of RAM
either. File Explorer will stop mallocing RAM, at around 15GB or so
on a 64-bit system.

The Recycle Bin is a FIFO cache: when its reserved space gets consumed,
it will delete the oldest deleted files until there is enough reserved
space for the new deleted file. It doesn't look at the total size of
all candidate delete files to first deleted all the oldest files to make
room (to make it one operation to removed deleted files all at once to
make room for all the newly deleted files). It does it on a one-to-one
basis: remove old deleted files until there is room for a new deleted
file, and repeat for each new deleted file in the pending list. This
will take LOTS of time. In fact, if you delete lots of files, some of
those just deleted might end up getting removed to make room for the
latest deleted files. Moving that view port of deleted files in the
Recycle Bin to keep the last N files within the reserve space incurs a
lot of overhead.

You could flush the Recycle Bin and hope the number of files you are
going to delete does not exceed the reserve space for the Recycle Bin.
Because I can't be sure how many of the old deleted files will get
removed, and because I likely don't care, I simply disabled the Recycle
Bin when deleting (moving) many thousands of files. A huge delete
operation (by file count) goes much faster when the Recycle Bin is
disabled.

When you say you disabled the Recycle Bin, is it safe to say that you
simply pressed Shift when you initiated the Delete operation, in order to
temporarily bypass the Recycle Bin?

That's one way. If I use Ctrl+A to pick all files, then Shift+Del works
well. If the selection is lost, it's easy to reselect the files.
However, if I'm going to be using Ctrl or Shift a lot to make separate
or ranged selections, especially if I use both, I don't like relying on
Shift+Del. What if I flub the Shift key and hit something else which
loses all those carefully selected files? I've done that. Far fewer is
when I've flubbed hitting just the Del key. If selections are easy,
like Ctrl+A, then Shift+Del is easy. If I'm making a hell of a lot of
individual file selections, multiple ranges of them, or both, I first
disable the Recycle Bin (pretty easy to turn off).

Fact is, the Recycle Bin is far more often in my way than when it helps.
Periodically I end up disabling it ... until when I delete something
and, oops, need it back. I probably undelete (move out) from the
Recycle Bin maybe twice per year. It also doesn't help when deleting
super-huge sized files (file too big for Recycle Bin on a volume). If
the Recycle Bin's algorithm were smarter to calculate the size needed
for the newly (proposed) deleted files and remove all at once the old
deleted files from its storage space, so it was faster, then I'd
probably not contemplate disabling the Recycle Bin. I don't know of a
tweak to make the Recycle Bin faster other than disabling it.

I've seen users complain of waiting half an hour just to empty the
Recycle Bin. Deleting is just moving the file. Emptying requires
removing each NFTS journal descriptor. Microsoft has not improved the
preformance of that safety feature.

If you're only interested in using the Recycle Bin to recover from your
own error when deleting a file, or two, then reducing its size helps.
However, if you delete several thousand files at a time then a small
reserve means most of those deleted files actually end up deleted and
only the last few remain. However, when deleting that many files, it's
possible a lot of the just-deleted files that were first in the list get
rolled out, too. I'll probably severely reduce the size of the Recycle
Bin since I'm only interested in it as a safety net when I delete a
file, or two. Yet, that still doesn't change its algorithm of getting
the file space needed for the current delete candidate and removing
enough old deleted files to make space for the new one, and then
repeating the process for each candidate delete file thereafter. It's a
crappy and ancient scheme.

I use CCleaner both manually via shortcut and as a scheduled task (using
the /auto command-line switch). For the temp folders, it has the option
"Only delete files in Windows Temp folders older than 24 hours". It has
a similar option for the Recycle Bin. That helps keep the temp folder
and Recycle Bin flushed of old entries that I'm likely not to need.
It's the immediate delete blunder from which I need to recover.

On Fri, 27 Sep 2019 01:56:57 -0500, VanguardLH wrote:

Fact is, the Recycle Bin is far more often in my way than when it helps.
Periodically I end up disabling it ... until when I delete something
and, oops, need it back. I probably undelete (move out) from the
Recycle Bin maybe twice per year. It also doesn't help when deleting
super-huge sized files (file too big for Recycle Bin on a volume). If
the Recycle Bin's algorithm were smarter to calculate the size needed
for the newly (proposed) deleted files and remove all at once the old
deleted files from its storage space, so it was faster, then I'd
probably not contemplate disabling the Recycle Bin. I don't know of a
tweak to make the Recycle Bin faster other than disabling it.

It may seem counterintuitive, but increasing the size of the Recycle Bin
helps a lot with the problems you're describing above. The downside is that
you'd probably want to adopt the practice of emptying the Bin on a regular
basis. You can either do that manually, or programmatically, or a
combination of both.

On my 33TB Data drive, the Recycle Bin is set to a custom size of 1.25TB. I
agree with you that the underlying Bin design is not optimized by any
means, but having a larger Bin at least means not having to deal with the
churn that you mentioned above.

On Fri, 27 Sep 2019 11:00:29 -0500, Char Jackson
wrote:

On my 33TB Data drive, the Recycle Bin is set to a custom size of 1.25TB. I


33TB? What drive is this.? The largest I've seen is 8TB.

On Fri, 27 Sep 2019 17:00:12 -0700, Ken Blake wrote:

On Fri, 27 Sep 2019 11:00:29 -0500, Char Jackson
wrote:

On my 33TB Data drive, the Recycle Bin is set to a custom size of 1.25TB. I


33TB? What drive is this.? The largest I've seen is 8TB.

It's 5 physical drives, managed as a single volume by a program called
DriveBender.

3x 4TB = 12TB
2x 12TB = 24TB
-------
Total 36TB

The filesystem is NTFS, so the usable space on the volume is 93.15% of the
total, or 33.5TB.

Another system here currently has a 40TB volume, also managed by
DriveBender. In that system, the drive pool consists of 10x 2TB plus 5x 4TB
for a total of 40TB, or 37.2TB usable.

As for individual drive sizes, 16TB seems to be the current max.
https://www.newegg.com/p/pl?N=100167523%20601334339

On Fri, 27 Sep 2019 21:33:37 -0500, Char Jackson
wrote:

On Fri, 27 Sep 2019 17:00:12 -0700, Ken Blake wrote:

On Fri, 27 Sep 2019 11:00:29 -0500, Char Jackson
wrote:

On my 33TB Data drive, the Recycle Bin is set to a custom size of 1.25TB. I


33TB? What drive is this.? The largest I've seen is 8TB.

It's 5 physical drives, managed as a single volume by a program called
DriveBender.

3x 4TB = 12TB
2x 12TB = 24TB
-------
Total 36TB



OK, thanks, I suspected something like that