If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
Several words on File Fragmentation and performance
I have always described the concept of file fragmentation as thus...
Say you are reading a Newspaper. You find an article on page 2 and read a couple of paragraphs and it then says "go to page 5". You go to page 5 and read another couple of paragraphs and it now says "go to page 21". You go to page 21 and read a few more paragraphs and it now says "go to page 12". You go to page 12 and read a few more paragraphs and the article is completed. Defragmentation is akin to taking all those paragraphs on other pages and assembling them all in one contiguous article on page 2. -- Dave "Leythos" wrote in message ... Over the last couple days we've had a thread about Fragmentation and Performance, as most of the long time administrators know how it impacts performance in the real world. For those of you that still are unsure, where are articles explaining it, even the mechanical aspect of it, for you to review. http://searchwindowssecurity.techtar.../0,293857,sid4 5_gci997657,00.html System performance and file fragmentation in Windows NT Date Published: 01 OCT 1999 About the White Paper: Contrary to early conventional wisdom about Windows NT, its file systems do become fragmented. This fragmentation occurs in the normal course of using the operating system. Theoretical analysis and real-world performance testing demonstrate that fragmentation has an adverse impact on system performance. Special characteristics of the NTFS file system, such as the paging file, directories and the Master File Table, are especially vulnerable to fragmentation, and allowing them to become fragmented is a guarantee of a decrease in overall system performance. Other NTFS features, such as file system compression, inherently create fragmentation. The best way to avoid these worst-case fragmentation problems, and to keep the system running at optimal performance, is to run a defragmentation system on a regularly scheduled basis. Both Windows NT workstations and NT servers are subject to these problems, and both can improve system performance through regular defragmentation. ********************* http://www.executive.com/whats-new/w...#_Toc463769971 NTFS Does Get Fragmented The Windows NTFS File System Driver uses a special file called the Master File Table (MFT) to track all files on that volume. The MFT starts out with some free space to allow new files to be tracked, but on a very busy system it too can run out of space. At this point NTFS extends the MFT itself, creating new stretches of it for new allocations. This situation is precipitated most often by fragmentation in the file system itself, as file system fragments consume entries in the MFT. If these new stretches are not contiguous, the MFT itself becomes fragmented. There are other files, such as the paging file used by Windows NT=3Fs virtual memory subsystem, which can also become fragmented with unpleasant implications for performance. The solution to these problems, as we will see, it to prevent them from happening by keeping your system defragmented. Lastly, directories in NTFS are allocated similarly to files, but defragmentation of them can be difficult. Performance Degradations Can Impede Productivity Windows NT does a good job of allowing the system to continue operation even as programs wait for disk I/O, but some inefficiency cannot be hidden forever. Especially on a mission-critical server, on which many users rely, inefficiencies in the file system can lead to performance degradation that impedes user productivity. These problems are not always apparent, and are frequently cavalierly blamed on other sources; perhaps the computer=3Fs just too slow, needs more memory, or some program being run needs an upgrade. Overall system performance is a complex phenomenon, and even experienced system administrators may not recognize fragmentation in a file system. After all, it can occur with large amounts of free space on the disk. But the main reason users don=3Ft recognize fragmentation is because Windows NT comes with no tools to identify it. Heavily used systems, which are by definition mission-critical systems for an organization, will become fragmented over time under normal usage in Windows NT. As performance decreases in such systems and users are forced to wait, productivity is thereby impeded. ********************* http://www.microsoft.com/windows2000...on/fileandprin t/defrag.asp File Fragmentation A file with all its parts stored in one location on a disk is described as "contiguous." If a file is not contiguous, it=3Fs fragmented; broken into pieces that are scattered throughout the disk. All Windows NT® and Windows 2000 file types=3FFile Allocation Table (FAT) and NTFS file system (NTFS)=3Fare susceptible to fragmentation. File fragmentation has a negative effect on disk performance because the disk head requires more time to move around to different points on the disk to read scattered file parts. This is a primary reason for the gradual degradation of system performance=3Fand the specific cause of longer reads and extended reboots. -- -- (Remove 999 to reply to me) |
Ads |
#2
|
|||
|
|||
Several words on File Fragmentation and performance
Leythos wrote:
There are other files, such as the paging file used by Windows NT=3Fs virtual memory subsystem, which can also become fragmented with unpleasant implications for performance. The solution to these problems, as we will see, it to prevent them from happening by keeping your system defragmented. Paging file fragmentation, while not totally irrelevant, is pretty much a non-issue simply because of the way that paging is done. Paging out is done based on lack of usage of each individual page (4K) and there is no requirement or attempt to ensure that pages moved out to the paging file are in any way related to each other. So if 100 pages (400k) is being moved from RAM to the paging file these 100 pages could be parts of a dozen or more different applications, device drivers, Windows components, or data files. If it is subsequently decided to move more items from RAM to the paging file the items being moved could again represent parts of many different items, and it is quite possible in fact even likely that some pages moved out in the second instance will be from the same items as were some of the pages moved out in the first instance. And then when one of these items becomes active again it will be necessary to reload the paged out pages for that item from the paging file. This could involve pages moved out in two or more different paging out operations and therefore the items being paged back in will not be contiguous even if the paging file is totally unfragmented. Ron Martell Duncan B.C. Canada -- Microsoft MVP On-Line Help Computer Service http://onlinehelp.bc.ca "The reason computer chips are so small is computers don't eat much." |
#3
|
|||
|
|||
Several words on File Fragmentation and performance
Leythos wrote:
Over the last couple days we've had a thread about Fragmentation and Performance, as most of the long time administrators know how it impacts performance in the real world. For those of you that still are unsure, where are articles explaining it, even the mechanical aspect of it, for you to review. http://searchwindowssecurity.techtar.../0,293857,sid4 5_gci997657,00.html System performance and file fragmentation in Windows NT Date Published: 01 OCT 1999 About the White Paper: Contrary to early conventional wisdom about Windows NT, its file systems do become fragmented. This fragmentation occurs in the normal course of using the operating system. Theoretical analysis and real-world performance testing demonstrate that fragmentation has an adverse impact on system performance. Special characteristics of the NTFS file system, such as the paging file, directories and the Master File Table, are especially vulnerable to fragmentation, and allowing them to become fragmented is a guarantee of a decrease in overall system performance. Other NTFS features, such as file system compression, inherently create fragmentation. The best way to avoid these worst-case fragmentation problems, and to keep the system running at optimal performance, is to run a defragmentation system on a regularly scheduled basis. Both Windows NT workstations and NT servers are subject to these problems, and both can improve system performance through regular defragmentation. ********************* http://www.executive.com/whats-new/w...#_Toc463769971 NTFS Does Get Fragmented The Windows NTFS File System Driver uses a special file called the Master File Table (MFT) to track all files on that volume. The MFT starts out with some free space to allow new files to be tracked, but on a very busy system it too can run out of space. At this point NTFS extends the MFT itself, creating new stretches of it for new allocations. This situation is precipitated most often by fragmentation in the file system itself, as file system fragments consume entries in the MFT. If these new stretches are not contiguous, the MFT itself becomes fragmented. There are other files, such as the paging file used by Windows NT=3Fs virtual memory subsystem, which can also become fragmented with unpleasant implications for performance. The solution to these problems, as we will see, it to prevent them from happening by keeping your system defragmented. Lastly, directories in NTFS are allocated similarly to files, but defragmentation of them can be difficult. Performance Degradations Can Impede Productivity Windows NT does a good job of allowing the system to continue operation even as programs wait for disk I/O, but some inefficiency cannot be hidden forever. Especially on a mission-critical server, on which many users rely, inefficiencies in the file system can lead to performance degradation that impedes user productivity. These problems are not always apparent, and are frequently cavalierly blamed on other sources; perhaps the computer=3Fs just too slow, needs more memory, or some program being run needs an upgrade. Overall system performance is a complex phenomenon, and even experienced system administrators may not recognize fragmentation in a file system. After all, it can occur with large amounts of free space on the disk. But the main reason users don=3Ft recognize fragmentation is because Windows NT comes with no tools to identify it. Heavily used systems, which are by definition mission-critical systems for an organization, will become fragmented over time under normal usage in Windows NT. As performance decreases in such systems and users are forced to wait, productivity is thereby impeded. ********************* http://www.microsoft.com/windows2000...on/fileandprin t/defrag.asp File Fragmentation A file with all its parts stored in one location on a disk is described as "contiguous." If a file is not contiguous, it=3Fs fragmented; broken into pieces that are scattered throughout the disk. All Windows NT® and Windows 2000 file types=3FFile Allocation Table (FAT) and NTFS file system (NTFS)=3Fare susceptible to fragmentation. File fragmentation has a negative effect on disk performance because the disk head requires more time to move around to different points on the disk to read scattered file parts. This is a primary reason for the gradual degradation of system performance=3Fand the specific cause of longer reads and extended reboots. Your first two references come from a seller of defrag software (hardly an unbiased source) and the third, from MS, includes nothing but unsupported assertions and irrelevant (to this discussion) generalizations. If this kind of thing impresses you, I have a good idea now how Dubya got reelected. |
#4
|
|||
|
|||
Several words on File Fragmentation and performance
Even though fragmentation occurs there are two amin cases where it matters
much less today than evne 3 years ago. 1. Hugh disks for the end user and very fast CPU and disks. 2. Servers have specific volumes for appiclation use, and data use. Only those ares of high volitility need to be defragment in the back ground, and depeneing on the volitility (20% change, 10% Add, 10% Delete) vs. 40% change, 25% Add, 20% change) may indicate that defraging will occurs naturely over time in a constance fragmented pattern. The world id different now, find another hobby. :P SJ "Leythos" wrote in message ... Over the last couple days we've had a thread about Fragmentation and Performance, as most of the long time administrators know how it impacts performance in the real world. For those of you that still are unsure, where are articles explaining it, even the mechanical aspect of it, for you to review. http://searchwindowssecurity.techtar.../0,293857,sid4 5_gci997657,00.html System performance and file fragmentation in Windows NT Date Published: 01 OCT 1999 About the White Paper: Contrary to early conventional wisdom about Windows NT, its file systems do become fragmented. This fragmentation occurs in the normal course of using the operating system. Theoretical analysis and real-world performance testing demonstrate that fragmentation has an adverse impact on system performance. Special characteristics of the NTFS file system, such as the paging file, directories and the Master File Table, are especially vulnerable to fragmentation, and allowing them to become fragmented is a guarantee of a decrease in overall system performance. Other NTFS features, such as file system compression, inherently create fragmentation. The best way to avoid these worst-case fragmentation problems, and to keep the system running at optimal performance, is to run a defragmentation system on a regularly scheduled basis. Both Windows NT workstations and NT servers are subject to these problems, and both can improve system performance through regular defragmentation. ********************* http://www.executive.com/whats-new/w...#_Toc463769971 NTFS Does Get Fragmented The Windows NTFS File System Driver uses a special file called the Master File Table (MFT) to track all files on that volume. The MFT starts out with some free space to allow new files to be tracked, but on a very busy system it too can run out of space. At this point NTFS extends the MFT itself, creating new stretches of it for new allocations. This situation is precipitated most often by fragmentation in the file system itself, as file system fragments consume entries in the MFT. If these new stretches are not contiguous, the MFT itself becomes fragmented. There are other files, such as the paging file used by Windows NT=3Fs virtual memory subsystem, which can also become fragmented with unpleasant implications for performance. The solution to these problems, as we will see, it to prevent them from happening by keeping your system defragmented. Lastly, directories in NTFS are allocated similarly to files, but defragmentation of them can be difficult. Performance Degradations Can Impede Productivity Windows NT does a good job of allowing the system to continue operation even as programs wait for disk I/O, but some inefficiency cannot be hidden forever. Especially on a mission-critical server, on which many users rely, inefficiencies in the file system can lead to performance degradation that impedes user productivity. These problems are not always apparent, and are frequently cavalierly blamed on other sources; perhaps the computer=3Fs just too slow, needs more memory, or some program being run needs an upgrade. Overall system performance is a complex phenomenon, and even experienced system administrators may not recognize fragmentation in a file system. After all, it can occur with large amounts of free space on the disk. But the main reason users don=3Ft recognize fragmentation is because Windows NT comes with no tools to identify it. Heavily used systems, which are by definition mission-critical systems for an organization, will become fragmented over time under normal usage in Windows NT. As performance decreases in such systems and users are forced to wait, productivity is thereby impeded. ********************* http://www.microsoft.com/windows2000...on/fileandprin t/defrag.asp File Fragmentation A file with all its parts stored in one location on a disk is described as "contiguous." If a file is not contiguous, it=3Fs fragmented; broken into pieces that are scattered throughout the disk. All Windows NT® and Windows 2000 file types=3FFile Allocation Table (FAT) and NTFS file system (NTFS)=3Fare susceptible to fragmentation. File fragmentation has a negative effect on disk performance because the disk head requires more time to move around to different points on the disk to read scattered file parts. This is a primary reason for the gradual degradation of system performance=3Fand the specific cause of longer reads and extended reboots. -- -- (Remove 999 to reply to me) |
#5
|
|||
|
|||
Several words on File Fragmentation and performance
David H. Lipman wrote:
I have always described the concept of file fragmentation as thus... Say you are reading a Newspaper. You find an article on page 2 and read a couple of paragraphs and it then says "go to page 5". You go to page 5 and read another couple of paragraphs and it now says "go to page 21". You go to page 21 and read a few more paragraphs and it now says "go to page 12". You go to page 12 and read a few more paragraphs and the article is completed. Defragmentation is akin to taking all those paragraphs on other pages and assembling them all in one contiguous article on page 2. That is a very nice analogy, Dave. Thank you -- Alex Nichol MS MVP (Windows Technologies) Bournemouth, U.K. (remove the D8 bit) |
#6
|
|||
|
|||
Several words on File Fragmentation and performance
Alex Nichol wrote:
David H. Lipman wrote: I have always described the concept of file fragmentation as thus... Say you are reading a Newspaper. You find an article on page 2 and read a couple of paragraphs and it then says "go to page 5". You go to page 5 and read another couple of paragraphs and it now says "go to page 21". You go to page 21 and read a few more paragraphs and it now says "go to page 12". You go to page 12 and read a few more paragraphs and the article is completed. Defragmentation is akin to taking all those paragraphs on other pages and assembling them all in one contiguous article on page 2. That is a very nice analogy, Dave. Thank you Actually, in the context of the discussion, it's not. While it serves to illustrate what happens in reading a fragmented file (and for that purpose it is indeed a good analogy) it is a bit misleading in this context, because paging around in a newspaper to read an article is a PITA, while the analogous action of the hard drive will probably be unnoticeable. |
#7
|
|||
|
|||
Several words on File Fragmentation and performance
On Thu, 23 Dec 2004 16:13:28 -0600, "Raymond J. Johnson Jr."
Actually, in the context of the discussion, it's not. While it serves to illustrate what happens in reading a fragmented file (and for that purpose it is indeed a good analogy) it is a bit misleading in this context, because paging around in a newspaper to read an article is a PITA, while the analogous action of the hard drive will probably be unnoticeable. Unless you are waiting for a modem, you are generally waiting for the HD, which is in effect not reading one file in several bits, but lots of files in several bits. So if fragmentation makes each one of those file acesses take longer, then yes; you could feel it. ---------- ----- ---- --- -- - - - - Proverbs Unscrolled #37 "Build it and they will come and break it" ---------- ----- ---- --- -- - - - - |
#8
|
|||
|
|||
Several words on File Fragmentation and performance
cquirke (MVP Win9x) wrote:
On Thu, 23 Dec 2004 16:13:28 -0600, "Raymond J. Johnson Jr." Actually, in the context of the discussion, it's not. While it serves to illustrate what happens in reading a fragmented file (and for that purpose it is indeed a good analogy) it is a bit misleading in this context, because paging around in a newspaper to read an article is a PITA, while the analogous action of the hard drive will probably be unnoticeable. Unless you are waiting for a modem, you are generally waiting for the HD, which is in effect not reading one file in several bits, but lots of files in several bits. So if fragmentation makes each one of those file acesses take longer, then yes; you could feel it. No one here can read, apparently. I NEVER SAID that fragmentation will NEVER cause noticeable problems. The idiot Leythos keeps trying to read that though, and now it looks like you are too. Here you go--let's see who can understand plain English: For most users, and "most users" does not include people who work with huge database or image files constantly, regular and habitual defragging is a waste of time. For MOST users, defragging should be done if the user is anal retentive and feels an irresistible compulsion to do it, or if there is a performance issue that might be ameliorated by defragging. |
#9
|
|||
|
|||
Several words on File Fragmentation and performance
On Thu, 23 Dec 2004 21:36:05 -0600, "Raymond J. Johnson Jr."
For most users, and "most users" does not include people who work with huge database or image files constantly, regular and habitual defragging is a waste of time. Most users wait for HD, and therefore will benefit from defragging. I do agree with you that regular defragging, when nothing much has changed on the system, is an ineffective way of managing this. I would defrag *only* if the PC is known to be reliable and stable, because to defrg on a flaky system is to risk installation or data loss. Even then, I'd usually do it only at these times: 1) After deleting or uninstalling a mass of stuff 2) Before installing or adding a mass of stuff 3) Follow-up defrag a week after changing core code Because Win98 and later track module usage during system use, to know what often-used code to locate for best speed, you may benefit from (3). For example, if you install a new version of IE, the new files aren't listed in AppLog (Win98/SE/ME) or Prefetch (XP) and thus won't be located for best speed. After a week's use, they will have been identified by the system to be particularly optimised. More effective strategies to reduce fragmentation impact include: a) Kill off massive web browser caches Web cache is supposed to speed up slow connections by holding most-recently-used web content. A connection that can populate a 256M cache within "recent" usage is fast enough not to need caching. A connection slow enough to need caching will take so long to populate more than 60M or so, that the oldest stuff won't be "recent" anymore. Web cache contains lots of small files, which are deleted when "old". If that takes months, you create fragmentation by opening up holes in past areas of use. Also, whenever IE has to check all names in a cache branch to ensure a new item doesn't overwrite and existing one, this will be slow on FATxx as it has to traverse a large directory that is itself fragmented. Ouch! So I usually limit cache to 20M. b) Keep C: small and lean By partitioning, and more to the point, being smart about what goes on which partition, you can keep a system running as fast with 60G of stuff on it as it was when there was only 10G of stuff there. When 90% of disk accesses (temp, swap, OS code, core apps) are in a C: that spans only 10% of the HD, your head travel will usually be no more than 10% of the span - no matter how fragged C: gets. Much smarter than having to step over your 40G MP3 collection, when stepping from first-installed OS code at the 'front" of C: to the fresh temp files created at the "end" of C:. The other payoff is reduced risk of data corruption, and faster maintenance. Files get corrupted during file system writes, and if your data is somewhere that avoids 90% of write traffic, it is that much safer. After a bad exit, you may typically find only C: has to be checked, and waiting for 8G to be checked is better than 60G. ---------- ----- ---- --- -- - - - - Proverbs Unscrolled #37 "Build it and they will come and break it" ---------- ----- ---- --- -- - - - - |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Defrag - Page file - NTFS questions | yandr | Hardware and Windows XP | 3 | December 14th 04 09:15 AM |
Managing XP system space for the MFT | Kelly | Hardware and Windows XP | 11 | August 19th 04 02:14 PM |
Managing XP system space for the MFT | DILIP | General XP issues or comments | 6 | August 19th 04 02:14 PM |
Managing XP system space for the MFT | Bob Harris | General XP issues or comments | 1 | August 18th 04 01:10 AM |
Managing XP system space for the MFT | Kelly | General XP issues or comments | 1 | August 14th 04 09:35 AM |