A Windows XP help forum. PCbanter

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

Go Back   Home » PCbanter forum » Windows 10 » Windows 10 Help Forum
Site Map Home Register Authors List Search Today's Posts Mark Forums Read Web Partners

Running Macrium destroys System Restore points



 
 
Thread Tools Rate Thread Display Modes
  #16  
Old July 8th 20, 04:13 PM posted to alt.comp.os.windows-10
Frank Slootweg
external usenet poster
 
Posts: 1,226
Default Running Macrium destroys System Restore points

Terry Pinnell wrote:
VanguardLH wrote:

[...]
However, if the OP is saving image backups, why waste the space to save
restore points?


Here are three reasons and I bet there are others:

1. SR is a complementary tool, not an alternative.

2. Simplicity. In a couple of clicks SR can often immediately fix an
accident or obscure issue.


Correction: System Restore *might* 'fix' an issue, but most likely
will - probably quietly - clobber something which it should not touch.

System Restore is broken-by-design. It *pretends* it knows what to
restore and what not, but it *cannot* possibly know that for non-MS
software. It also pretends it doesn't touch user data, but because it
happily clobbers data in folders where it has no business and pretends
it knows what to restore and what not in those folders, it's not just
broken-by-design, but lying as well.

System Restore, just say no!

Some references:

'About System Restore'
https://docs.microsoft.com/en-us/windows/win32/sr/about-system-restore
lie
"System Restore does not restore user data or documents, so it will not
cause users to lose their files, e-mail, browsing history, or
favorites."
/lie

'Monitored File Name Extensions'
http://msdn.microsoft.com/en-us/library/windows/desktop/aa378870(v=vs.85).aspx

Some examples of System Restore clobbering *user* data:

- SR will restore/clobber .old and .bk1 but not .bck. It shouldn't touch
any of these in user folders.
I had a Windows Mail folder (looks like "e-mail" doesn't it?) - i.e.
*user* data - called Inbox.old, which SR would happily clobber.
AFAICT, .bk1 is not used by Microsoft, so why restore/clobber it?

- SR will restore/clobber .data, but not .dat. It shouldn't touch either
in user folders.
N.B. .dat is used by Internet Explorer and Windows itself (so
restore), but also by avast! (so don't restore). Did I already say
broken-by-design?

- SR will happily restore/clobber .exe files in *non*-Microsoft (read:
user) directories (such as \home\franks).

- My local News server uses .ini files which contains pointers to .dat
files which contain the data (articles). I.e. a simple database per
newsgroup.
SR happily restores/clobbers the .ini files, but not the .dat files,
causing it to not only to restore user data it says it doesn't touch,
but also happily corrupting the databases, i.e. they are neither old
nor current, but corrupt.

Need I say more?

Bottom line: System Restore *cannot* work correctly, hence it's
broken-by-design.
Ads
  #17  
Old July 9th 20, 01:18 AM posted to alt.comp.os.windows-10
VanguardLH[_2_]
external usenet poster
 
Posts: 10,881
Default Running Macrium destroys System Restore points

When you created the backup jobs, did you ONLY use Reflect's GUI to
define them? Or did you use its ability to create batch, VBscript, or
Powershell scripts? Reflect (in the Home edition that I have, don't
know about pro or enterprise editions) does not let you define pre- and
post-commands for a backup job. Those would be added commands to run
other software either before or after a backup job. For example, a
pre-command that runs CCleaner (ccleaner.exe /auto) before Reflect
begins its backup job would delete garbage not to include in the backup
(and Reflect, by default, skips the temp folders). Since Reflect has no
pre/post-command feature, the only way to effect that feature is to have
Reflect create a script file that you could then edit to add your own
commands in the order you want relative to the command that starts the
backup job.

Update: I was wrong about pre/post-commands. Reflect does not let you
define pre/post commands but not on a per-backup basis. It has global
settings for pre/post commands. In the menu bar, go to Other Tasks -
Edit Defaults - Scripts. Select "Run Programs" in left pane. Are
both "script at the start" and "script at the end" options deselected?

Did you define your own scripts, like in a batch file, to run the
command for Reflect along with any other commands? Looks like you
scheduled the backup jobs. Reflect uses the Task Scheduler service
already provided by Windows (instead of its own scheduler service, like
some other backup programs). What is the command specified in the
Actions tab for the Macrium event(s) defined in Task Scheduler?

Did you check the configured size for reserve space for your restore
points? How big is the reserved space? Reflect has to store somewhere
the shadows created by the VSS service. You need to make the SR
reserved space as big as the number of SR points you want to retain
*AND* to hold any other VSS snapshots (aka shadows). Old snapshots get
rolled out to make room within the allocated reserved space to make room
for new snapshots. The bigger the reserved space, the more snapshots
can be retained. However, bigger snapshots will also cause less older
snapshots to be retained. One huge snapshot/shadow could end up wiping
out many older and smaller snapshots.

As I recall, the recommendation or default is to allocate 10% of the
drive's size to SR reserved storage. The space is reserved, not
immediately allocated. Shadows can grow to accumulate that much space,
and then old ones roll out to let in new ones. If snapshots are
deleted, like all of them, and the user consumes the free space on the
drive, there is no space left to allocate later for the reserved storage
space for snapshots.

Aside: Do you have any other cleaners scheduled to run about the same
time as your Reflect backup? For example, Disk Cleanup (cleanmgr.exe)
that comes with Windows can delete all but the last restore point.

Paul mentioned using "vssadmin list shadows" to show stored shadow
copies. I didn't think those were used for System Restore files, but
later reading shows they are. That makes sense. A program, like a
backup program, that creates a shadow copy only needs it temporarily,
like during the backup job, and that shadow should get deleted when the
backup is done. However, System Restore also uses VSS and stores the
shadows (not full partition, but just some folders and files) as restore
points, so it makes sense that "vssadmin list shadows" would also list
the restore points that have been saved by SR. Temporary shadows
created by VSS-enabled backup programs should disappear when the backup
job completes because the backup image file is used for restore, not the
old shadow copy. Besides listing shadows kept as restore points, you
can also delete them using vssadmin. When you run "vssadmin list
shadows", is only one listed after the Reflect full backup job? Before
you run Reflect, were more than 1 shadow listed by vssadmin?

Back to the reserved size for SR points, apparently that is the maximum
size for all saved shadows. Although Reflect creates a temporary shadow
(disappears after use), but it still exists during the backup and still
consumes a portion of the reserved SR space. The more write activity
there is on a drive during a backup, the larger the shadow becomes
(copy-on-write behavior). During a backup, and writing a lot to that
drive, will push up the size of the shadow. So, I can see a shadow,
even a temporary one, getting so large that it pushes out the oldest
shadows from the SR reserved space that has a max capacity. Have you
tried upping the size of the SR storage space, like go twice its size?
Is more than one drive included in the backup job? The SR space might
be sufficient on C: to accomodate the new temporary shadow created by
VSS on behalf of Reflect, but maybe the SR space is too small on other
drives.

https://helpdeskgeek.com/wp-content/...10/image22.png

That shows the dialog where you decide on which drives to enable SR.
For example, if you have an internal HDD or SDD to where you store local
copies of the backup files (in addition to saving them to removable
media, or, at least, have another copy of them on a different physical
disk), you wouldn't want SR enabled there. There would be no point to
create SR restore points on a drive with no OS and the files are
themselves backups but would never get included in an SR snapshot.
Select a drive where SR is enabled, and then click on the Configure
button. Each drive where enabled has the snapshots stored under the
"System Volume Information" folder, so the shadows are saved on the same
drive that they cover. SR incurs a lot of writing, so you don't want it
enabled on an SSD drive (another reason not to use SR, especially if
using daily scheduled backups saved to other drives or removable media).
What is the current Max Usage size (shadow storage space)? Could be it
is too low. Alas, the setting is by percent and global across all
drives where SR is enabled, so setting a much later size for a huge
drive also means reserving the same amount of space on smaller drives.
However, remember that the space is reserved and only consumed when
needed. If you are creating lots of snaphots, you could end up filling
up to the max size of the reserved space and rolling out old SR points.
Creating huge snapshots can also mean rolling out old snapshots.
Writing to the drive while backing it up will create a much larger
snapshot, and you're doing a full backup of 1, or perhaps more, drives.

Is the computer quiescent during the backup job's scheduled run time?
That is, other than services or backgrounded processes (like startup
programs that remain loaded) that don't do much writing to the drive, is
anything else left running, scheduled to run, or that you are running
that will generate lots of writes to the same drive that is getting
backed up?

I was thinking that perhaps you could open an elevated command shell in
which to repeatedly run "vssadmin list shadows" to watch the size of the
shadow created by Reflect. That is, open an elevated command shell, run
"vssadmin list shadows" (to put in the command cache), run the full
backup job, and in the command shell use the up-arrow key to redisplay
the same command and hit Enter, so you could monitor the size of that
shadow. Alas, the list operator does not show the current size of the
shadow. I did find another vssadmin operation that shows size:

vssadmin list shadowstorage

That shows the configured max size for reserved shadow storage space
(where SR and Reflect are storing their shadows) along with its current
usage (allocation size). So, maybe running a full backup and monitoring
the consumption of shadow storage space might indicate if Reflect's
shadow is squeezing out older shadows. If so, you need to up the size
of SR storage using the GUI method or "vssadmin Resize ShadowStorage" in
an elevated command shell. There might be a shadow storage logger to
keep a history of how the reserved space was getting used, which shadows
are and were saved in it, and show how large were each of the snapshots.
Having to manually re-run the above vssadmin command in an elevated
command shell is tedious, requires you to do it while the backup job is
running, and is hard to decipher regarding how shadow storage space was
getting consumed and when there were changes.

By the way, although it has been mentioned old snapshots get rolled out
when new ones are created that need more space than is currently
available in the reserved storage space, that's not entirely how old
snapshots are deleted. Besides manual deletion (via vssadmin, a cleaner
tool, tweaker, or other means), the OS will start deleting old snapshots
when the SR storage space reaches 90% usage. It does this to make room
for later snapshot storage under the presumption that a later snapshot
won't be larger than the remaining 10% of the reserved space, but that
really depends on how much you allocated to the reserved space in the
first place. Upping the SR storage space, like twice its current size,
might show if some old snapshots hang around after Reflect's full backup
job completes (since the shadow could grow during the backup job due to
writes to the same drive).

The problem might be you just don't have a big enough allocated size to
the reserved space for the SR storage space. Any big snapshot could end
up rolling out lots of older snapshots.
  #18  
Old July 9th 20, 01:35 AM posted to alt.comp.os.windows-10
VanguardLH[_2_]
external usenet poster
 
Posts: 10,881
Default Running Macrium destroys System Restore points

Oh, and adding to how old snapshots get deleted which, to you, seems
premature, the growth in usage of the shadow storage by a snapshot may
outstrip how fast the VSS service can write (and enlarge) the snapshot.

https://social.technet.microsoft.com...=windowsbackup

Apparently when shadows get deleted, an event gets logged. I'm not sure
event ID 25 by the VolSnap source is the only one that records deletion
of snapshots.

https://support.unitrends.com/Unitre...icle/000003818

According to that article (a copy of an MS KB article), the
VSS_E_MAXIMUM_NUMBER_OF_SNAPSHOTS_REACHED error means there is some
maximum number of snapshots that can be written to the shadow storage
space.

Since the storage space is reserved and not immediately allocated,
another possibility is there is not currently enough free space
available on the drive to allow growing the size of the SR storage
space. You might have a huge size configured for the SR storage space,
but if there is not enough free space on the drive then the SR storage
space cannot grow which means it has to delete old snapshots to make
room for a new one.

What is the configured max size for the reserved SR storage space?
Multiply that percentage by the size of the drive (partition or volume)
that is getting included in the full backup job. Then check if there is
that much free space, or more, left on the drive. Just upping the max
size of SR storage space will have no effect on the free space available
on the drive. That's probably one of the reasons why SR storage space
is, by default, about 5% of the drive's size but it is recommended you
keep 10%, or more, of the drive's capacity as free space. There needs
to be room for more snapshots, room for the usage to grow for the SR
storage space on an as-needed basis, room for temp files, pagefile, and
yadda yadda to make sure everything keeps running. Getting low on free
disk space, and even before Windows alerts you with its "low on disk
space" alert, causes problems. You could delete programs, install them
on other drives, remove data files to other drives or removable media,
and other cleanup to up the free space, but that just masks and prolongs
the real issue that you need a bigger disk.
  #20  
Old July 9th 20, 01:11 PM posted to alt.comp.os.windows-10
Steve
external usenet poster
 
Posts: 1
Default Running Macrium destroys System Restore points

On Tue, 07 Jul 2020 22:39:21 -0500, Ant wrote:

???? Good Guy ???? wrote:
[-- text/plain, encoding 8bit, charset: utf-8, 8 lines --]


You are not authorised to read my posts in plain text. Please install HTML enabled newsreader, such as latest Thunderbird https://www.thunderbird.net, to benefit from solutions posted in my posts.


Why no plain text format?


It's self censorship.
Honestly, he's doing you a big favour.
  #21  
Old July 11th 20, 06:23 PM posted to alt.comp.os.windows-10
Terry Pinnell[_3_]
external usenet poster
 
Posts: 732
Default Running Macrium destroys System Restore points

VanguardLH wrote:

When you created the backup jobs, did you ONLY use Reflect's GUI to
define them?


Yes. It's just a single, weekly, full backup as shown in my previous
screenshot.

Or did you use its ability to create batch, VBscript, or
Powershell scripts? Reflect (in the Home edition that I have, don't
know about pro or enterprise editions) does not let you define pre- and
post-commands for a backup job. Those would be added commands to run
other software either before or after a backup job. For example, a
pre-command that runs CCleaner (ccleaner.exe /auto) before Reflect
begins its backup job would delete garbage not to include in the backup
(and Reflect, by default, skips the temp folders). Since Reflect has no
pre/post-command feature, the only way to effect that feature is to have
Reflect create a script file that you could then edit to add your own
commands in the order you want relative to the command that starts the
backup job.

Update: I was wrong about pre/post-commands. Reflect does not let you
define pre/post commands but not on a per-backup basis. It has global
settings for pre/post commands. In the menu bar, go to Other Tasks -
Edit Defaults - Scripts. Select "Run Programs" in left pane. Are
both "script at the start" and "script at the end" options deselected?

Did you define your own scripts, like in a batch file, to run the
command for Reflect along with any other commands? Looks like you
scheduled the backup jobs. Reflect uses the Task Scheduler service
already provided by Windows (instead of its own scheduler service, like
some other backup programs). What is the command specified in the
Actions tab for the Macrium event(s) defined in Task Scheduler?

Did you check the configured size for reserve space for your restore
points? How big is the reserved space? Reflect has to store somewhere
the shadows created by the VSS service. You need to make the SR
reserved space as big as the number of SR points you want to retain
*AND* to hold any other VSS snapshots (aka shadows). Old snapshots get
rolled out to make room within the allocated reserved space to make room
for new snapshots. The bigger the reserved space, the more snapshots
can be retained. However, bigger snapshots will also cause less older
snapshots to be retained. One huge snapshot/shadow could end up wiping
out many older and smaller snapshots.

As I recall, the recommendation or default is to allocate 10% of the
drive's size to SR reserved storage. The space is reserved, not
immediately allocated. Shadows can grow to accumulate that much space,
and then old ones roll out to let in new ones. If snapshots are
deleted, like all of them, and the user consumes the free space on the
drive, there is no space left to allocate later for the reserved storage
space for snapshots.


I currently have it set to 5% (11.5 GB) but will try 10%.

Aside: Do you have any other cleaners scheduled to run about the same
time as your Reflect backup? For example, Disk Cleanup (cleanmgr.exe)
that comes with Windows can delete all but the last restore point.

No.

Paul mentioned using "vssadmin list shadows" to show stored shadow
copies. I didn't think those were used for System Restore files, but
later reading shows they are. That makes sense. A program, like a
backup program, that creates a shadow copy only needs it temporarily,
like during the backup job, and that shadow should get deleted when the
backup is done. However, System Restore also uses VSS and stores the
shadows (not full partition, but just some folders and files) as restore
points, so it makes sense that "vssadmin list shadows" would also list
the restore points that have been saved by SR. Temporary shadows
created by VSS-enabled backup programs should disappear when the backup
job completes because the backup image file is used for restore, not the
old shadow copy. Besides listing shadows kept as restore points, you
can also delete them using vssadmin. When you run "vssadmin list
shadows", is only one listed after the Reflect full backup job? Before
you run Reflect, were more than 1 shadow listed by vssadmin?

Back to the reserved size for SR points, apparently that is the maximum
size for all saved shadows. Although Reflect creates a temporary shadow
(disappears after use), but it still exists during the backup and still
consumes a portion of the reserved SR space. The more write activity
there is on a drive during a backup, the larger the shadow becomes
(copy-on-write behavior). During a backup, and writing a lot to that
drive, will push up the size of the shadow. So, I can see a shadow,
even a temporary one, getting so large that it pushes out the oldest
shadows from the SR reserved space that has a max capacity. Have you
tried upping the size of the SR storage space, like go twice its size?

Yes, see above.

Is more than one drive included in the backup job?

No

The SR space might
be sufficient on C: to accomodate the new temporary shadow created by
VSS on behalf of Reflect, but maybe the SR space is too small on other
drives.

https://helpdeskgeek.com/wp-content/...10/image22.png

That shows the dialog where you decide on which drives to enable SR.
For example, if you have an internal HDD or SDD to where you store local
copies of the backup files (in addition to saving them to removable
media, or, at least, have another copy of them on a different physical
disk), you wouldn't want SR enabled there. There would be no point to
create SR restore points on a drive with no OS and the files are
themselves backups but would never get included in an SR snapshot.
Select a drive where SR is enabled, and then click on the Configure
button. Each drive where enabled has the snapshots stored under the
"System Volume Information" folder, so the shadows are saved on the same
drive that they cover. SR incurs a lot of writing, so you don't want it
enabled on an SSD drive (another reason not to use SR, especially if
using daily scheduled backups saved to other drives or removable media).
What is the current Max Usage size (shadow storage space)? Could be it
is too low. Alas, the setting is by percent and global across all
drives where SR is enabled, so setting a much later size for a huge
drive also means reserving the same amount of space on smaller drives.
However, remember that the space is reserved and only consumed when
needed. If you are creating lots of snaphots, you could end up filling
up to the max size of the reserved space and rolling out old SR points.
Creating huge snapshots can also mean rolling out old snapshots.
Writing to the drive while backing it up will create a much larger
snapshot, and you're doing a full backup of 1, or perhaps more, drives.

Is the computer quiescent during the backup job's scheduled run time?
That is, other than services or backgrounded processes (like startup
programs that remain loaded) that don't do much writing to the drive, is
anything else left running, scheduled to run, or that you are running
that will generate lots of writes to the same drive that is getting
backed up?

I was thinking that perhaps you could open an elevated command shell in
which to repeatedly run "vssadmin list shadows" to watch the size of the
shadow created by Reflect. That is, open an elevated command shell, run
"vssadmin list shadows" (to put in the command cache), run the full
backup job, and in the command shell use the up-arrow key to redisplay
the same command and hit Enter, so you could monitor the size of that
shadow. Alas, the list operator does not show the current size of the
shadow. I did find another vssadmin operation that shows size:

vssadmin list shadowstorage

That shows the configured max size for reserved shadow storage space
(where SR and Reflect are storing their shadows) along with its current
usage (allocation size). So, maybe running a full backup and monitoring
the consumption of shadow storage space might indicate if Reflect's
shadow is squeezing out older shadows. If so, you need to up the size
of SR storage using the GUI method or "vssadmin Resize ShadowStorage" in
an elevated command shell. There might be a shadow storage logger to
keep a history of how the reserved space was getting used, which shadows
are and were saved in it, and show how large were each of the snapshots.
Having to manually re-run the above vssadmin command in an elevated
command shell is tedious, requires you to do it while the backup job is
running, and is hard to decipher regarding how shadow storage space was
getting consumed and when there were changes.

By the way, although it has been mentioned old snapshots get rolled out
when new ones are created that need more space than is currently
available in the reserved storage space, that's not entirely how old
snapshots are deleted. Besides manual deletion (via vssadmin, a cleaner
tool, tweaker, or other means), the OS will start deleting old snapshots
when the SR storage space reaches 90% usage. It does this to make room
for later snapshot storage under the presumption that a later snapshot
won't be larger than the remaining 10% of the reserved space, but that
really depends on how much you allocated to the reserved space in the
first place. Upping the SR storage space, like twice its current size,
might show if some old snapshots hang around after Reflect's full backup
job completes (since the shadow could grow during the backup job due to
writes to the same drive).

The problem might be you just don't have a big enough allocated size to
the reserved space for the SR storage space. Any big snapshot could end
up rolling out lots of older snapshots.


Thanks for that generously comprehensive analysis.

A few days ago I decided to confirm that a MANUAL backup would still
delete all SRs, as I fully expected.
https://www.dropbox.com/s/tmpnmt10yg...alMR.jpg?raw=1

It didn't! That single SR point was still there. And it was still
present after a PC reset. That does seem to point to something special
going on around 05:00 on Sunday mornings. So I'm getting closer to a
solution. I've left the SR setting at 5% for now, but changed the start
time of the MR run to 06:00 tomorrow morning.

I'll report back with results.

Terry, East Grinstead, UK
  #22  
Old July 14th 20, 04:11 PM posted to alt.comp.os.windows-10
Terry Pinnell[_3_]
external usenet poster
 
Posts: 732
Default Running Macrium destroys System Restore points

Terry Pinnell wrote:


Thanks for that generously comprehensive analysis.

A few days ago I decided to confirm that a MANUAL backup would still
delete all SRs, as I fully expected.
https://www.dropbox.com/s/tmpnmt10yg...alMR.jpg?raw=1

It didn't! That single SR point was still there. And it was still
present after a PC reset. That does seem to point to something special
going on around 05:00 on Sunday mornings. So I'm getting closer to a
solution. I've left the SR setting at 5% for now, but changed the start
time of the MR run to 06:00 tomorrow morning.

I'll report back with results.


Still no idea why, but happy to report that after only that single
change all my SR points were preserved!

Of course, if I was being really thorough, I'd revert to 05:00 and check
again next Sunday, to eliminate the possibility that it was some other
(unknown) cause that has somehow fixed itself. Curiosity tempts me - but
I'll pass.

Terry
 




Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off






All times are GMT +1. The time now is 02:48 AM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 PCbanter.
The comments are property of their posters.