If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
|
Thread Tools | Rate Thread | Display Modes |
#1
|
|||
|
|||
Macrium scheduling
Some background before I get to my specific question.
I setup Macrium Reflect (Free) over three years ago after helpful advice here. After a few months of occasional checks I left it to do its stuff and essentially forgot about it. It was scheduled very early every Saturday morning so I didn't often get to see it start anyway. But yesterday I discovered that every weekly run since May had been unsuccessful ;-( (The reason turned out to be that somehow the drive letter of my destination external USB drive letters had been altered.) -------------------- In the course of reconfiguring I'm therefore reconsidering my choice of various scheduling templates, I think my previous was 'Incremental Backup Set', and that now seems perhaps a bit over complicated for a solo end user with no network stuff? I'd appreciate others' opinions or experience please. METHODS #1 Grandfather, Father, Son: Daily Incremental ("Son"), weekly Differential ("Father"), and monthly Full ("Grandfather") backups. #2 Differential Backup Set: A Full backup is created periodically followed by daily Differential backups. #3 Incremental Backup Set: A Full backup is created periodically followed by daily Incremental backups. #4 Incrementals Forever: Incrementals forever optimizes backup space and time by only ever creating a single Full backup. After this Incremental backups are created ad infinitum. The Full backup is consolidated with subsequent Incremental backups once the specified number of Incremental backups is reached. This is also known as a Synthetic Full backup. -------------------- As well as the 'method', any tips on optimal sequence of the full and incremental runs? Oh, and how often do others make a new recovery USB? Terry, East Grinstead, UK |
Ads |
#2
|
|||
|
|||
Macrium scheduling
On 11/19/19 6:27 AM, Terry Pinnell wrote:
Some background before I get to my specific question. I setup Macrium Reflect (Free) over three years ago after helpful advice here. After a few months of occasional checks I left it to do its stuff and essentially forgot about it. It was scheduled very early every Saturday morning so I didn't often get to see it start anyway. But yesterday I discovered that every weekly run since May had been unsuccessful ;-( (The reason turned out to be that somehow the drive letter of my destination external USB drive letters had been altered.) -------------------- In the course of reconfiguring I'm therefore reconsidering my choice of various scheduling templates, I think my previous was 'Incremental Backup Set', and that now seems perhaps a bit over complicated for a solo end user with no network stuff? I'd appreciate others' opinions or experience please. METHODS #1 Grandfather, Father, Son: Daily Incremental ("Son"), weekly Differential ("Father"), and monthly Full ("Grandfather") backups. #2 Differential Backup Set: A Full backup is created periodically followed by daily Differential backups. #3 Incremental Backup Set: A Full backup is created periodically followed by daily Incremental backups. #4 Incrementals Forever: Incrementals forever optimizes backup space and time by only ever creating a single Full backup. After this Incremental backups are created ad infinitum. The Full backup is consolidated with subsequent Incremental backups once the specified number of Incremental backups is reached. This is also known as a Synthetic Full backup. -------------------- As well as the 'method', any tips on optimal sequence of the full and incremental runs? Oh, and how often do others make a new recovery USB? Terry, East Grinstead, UK I do full backups. HD space is cheap and I have plenty. My backup is 40GB and I can get 5 easily on my 1TB drive. I find it easier if I have to reload to know that I don't have to jump through hoops to get a proper load. Truthfully I'm not really sure how incremental restores work. Do you have to restore A then B the C etc or does the software do it automatically? For this reason I just opted for full every time. I guess I opted for the lazy way out and not do research. I use Acronis so the restore of incremental may be different than Macrium. With Acronis I can restore a single file or single partition even though its a full backup. So I have the flexibility I need. Al |
#3
|
|||
|
|||
Macrium scheduling
Terry Pinnell wrote:
Some background before I get to my specific question. I setup Macrium Reflect (Free) over three years ago after helpful advice here. After a few months of occasional checks I left it to do its stuff and essentially forgot about it. It was scheduled very early every Saturday morning so I didn't often get to see it start anyway. But yesterday I discovered that every weekly run since May had been unsuccessful ;-( (The reason turned out to be that somehow the drive letter of my destination external USB drive letters had been altered.) -------------------- In the course of reconfiguring I'm therefore reconsidering my choice of various scheduling templates, I think my previous was 'Incremental Backup Set', and that now seems perhaps a bit over complicated for a solo end user with no network stuff? I'd appreciate others' opinions or experience please. METHODS #1 Grandfather, Father, Son: Daily Incremental ("Son"), weekly Differential ("Father"), and monthly Full ("Grandfather") backups. #2 Differential Backup Set: A Full backup is created periodically followed by daily Differential backups. #3 Incremental Backup Set: A Full backup is created periodically followed by daily Incremental backups. #4 Incrementals Forever: Incrementals forever optimizes backup space and time by only ever creating a single Full backup. After this Incremental backups are created ad infinitum. The Full backup is consolidated with subsequent Incremental backups once the specified number of Incremental backups is reached. This is also known as a Synthetic Full backup. -------------------- As well as the 'method', any tips on optimal sequence of the full and incremental runs? Oh, and how often do others make a new recovery USB? Terry, East Grinstead, UK Since Uwe was along today, in his honor we can promote this program. It's a Drive Letter Manager to help with USB devices. https://www.uwe-sieber.de/usbdlm_e.html ******* #2 and #3 are part of implementing #1. #4 is a fancy version of #3. So that leaves a discussion of #1. #1 starts with a Full, followed by incrementals. You can have as many incrementals as you want, but they have to remain readable. If you have 200 of them, and the third one croaks, you have a large pile of unusable incrementals and a very poor outcome. The incremental chain must be truncated, by doing a Full every once in a while. Then the incrementals start again, being applied to that Full. This is a reliability issue. If you keep 200 Incrementals with respect to a Full, all 200 have to be readable when you "restore to the 200th one". Macrium (as it should), treats a Full and a set of Incrementals as a "complete set". If you need to free up space, it generally deletes a Full plus the associated Incrementals, as a set. So what does the "Father" do ? since it's a differential, it works out the differences since the original Full. It allows the incrementals to be deleted, and more incrementals to be run. At the end of the week, another Differential is created, and the incrementals erased (since the Differential captured everything they did, and now the Incrementals are redundant). The Differentials are points in time, so if we need to go back a week, we can. The incrementals give us one day resolution (but only for the last week). While the differentials allow us to go back a week at a time only. When the end of the month arrives, a Full is made. This allows the previous Full and the Differentials to be erased if space is needed. The Fulls collected each month, allow us to go back a month for each one. But they're likely to be large files, so keeping 12 of those would be a handful. At some point, the Capacity Manager thing is going to start throwing month fulls, overboard. If I did that sequence on this machine, my backup drive would be exhausted by January with that pattern. The incrementals and weekly differentials wouldn't be that big of a burden, but collecting Fulls once a month will add up. (Twelve times 2TB would be 24TB per year.) The Incrementals Forever idea, is to have the efficiency of Incrementals (really short backup interval), with the capping off using a synthetic full, for some period of time. But by itself, #4 isn't a complete strategy, whereas #1 is closer to a complete strategy (a strategy the IT department might use, with some adjustments). Once the monthly Fulls are made by #1, you have the option of moving them off the drive to make room for the pattern. For example, since my backup drive could not handle too many Fulls collected once a month, when I hit January, I might move November to another drive. Leaving December and January on there. If I were to delete the current Full, then all the associated Incrementals and Differentials should disappear at the same instant. Whereas if I throw away the July Full, it is independent of what is happening in November. If you're going to adopt #1, I'd probably want at least room for two Fulls, and then move the excess (old) Fulls somewhere else. Once you have 24 Fulls in hand, you could keep the oldest one and the twelth one, and those are your yearly backups. Then keep monthly ones that the sequence is generating. For example, I once had to go back two years, to discover when my drive got a certain "unremovable driver" added to it. So old backups come in handy, "once in a lifetime". If I only kept those with a temporal resolution of "once a year", I probably wouldn't be that upset. So while #1 is the correct sequence, it hasn't made allowance for a yearly plan as such. You can do that manually of course. You can add more "tiers" to the plan, and make it a bit more sensible with respect to the size of the backup drive. For example, I don't really need monthly ones forever, and after three months, I could keep ones from every quarter and from every year after that. I would need some kind of RAID array, to make a pool big enough for #1 without modification :-) If you had a dinky C: and a huge external, then perhaps #1 could run for quite a while without intervention. Paul |
#4
|
|||
|
|||
Macrium scheduling
Big Al wrote:
On 11/19/19 6:27 AM, Terry Pinnell wrote: Some background before I get to my specific question. I setup Macrium Reflect (Free) over three years ago after helpful advice here. After a few months of occasional checks I left it to do its stuff and essentially forgot about it. It was scheduled very early every Saturday morning so I didn't often get to see it start anyway. But yesterday I discovered that every weekly run since May had been unsuccessful ;-( (The reason turned out to be that somehow the drive letter of my destination external USB drive letters had been altered.) -------------------- In the course of reconfiguring I'm therefore reconsidering my choice of various scheduling templates, I think my previous was 'Incremental Backup Set', and that now seems perhaps a bit over complicated for a solo end user with no network stuff? I'd appreciate others' opinions or experience please. METHODS #1 Grandfather, Father, Son: Daily Incremental ("Son"), weekly Differential ("Father"), and monthly Full ("Grandfather") backups. #2 Differential Backup Set: A Full backup is created periodically followed by daily Differential backups. #3 Incremental Backup Set: A Full backup is created periodically followed by daily Incremental backups. #4 Incrementals Forever: Incrementals forever optimizes backup space and time by only ever creating a single Full backup. After this Incremental backups are created ad infinitum. The Full backup is consolidated with subsequent Incremental backups once the specified number of Incremental backups is reached. This is also known as a Synthetic Full backup. -------------------- As well as the 'method', any tips on optimal sequence of the full and incremental runs? Oh, and how often do others make a new recovery USB? Terry, East Grinstead, UK I do full backups. HD space is cheap and I have plenty. My backup is 40GB and I can get 5 easily on my 1TB drive. I find it easier if I have to reload to know that I don't have to jump through hoops to get a proper load. Truthfully I'm not really sure how incremental restores work. Do you have to restore A then B the C etc or does the software do it automatically? For this reason I just opted for full every time. I guess I opted for the lazy way out and not do research. I use Acronis so the restore of incremental may be different than Macrium. With Acronis I can restore a single file or single partition even though its a full backup. So I have the flexibility I need. Al One full plus one differential, restores a differential to the desired date. At any time, all we need is the two files, to have integrity. But, the differential scheme wastes space (compared to incremental). One full plus *every* incremental to the desired date, gets you back to the desired date. If you chain too many incrementals together, the conditional probability of a "problem" with just one of them, becomes too great for a mathematical notion of reliability. If you chain 200 of them together, and "try to restore to today", one of those is bound to have a "read error" and then you're cooked with respect to restoring to today. You shouldn't set the incremental pool size too large, as a result. One could argue you're in just as much danger, by using one and only one backup drive connected externally to the machine, since that has everything on it, and is a single point of failure. Moving files off it, or duplicating it, might make a purist feel better. Duplicating the backup drive, has as much to do with "protecting you from your own stupidity", as anything related to hardware. Clicking the wrong button in the backup software, might knock out a set of backups. Or the Capacity Manager might erase things that you decide later, you would rather have kept. You can either move the stuff off, that is safe to move (the bits that won't affect the current state machine), or you can duplicate the whole thing once in a while. And leave the second device disconnected, and sitting in the corner for emergencies. The possibilities are endless... and expensive. Paul |
#5
|
|||
|
|||
Macrium scheduling
On 11/19/19 8:50 AM, Paul wrote:
YouÂ*canÂ*eitherÂ*moveÂ*theÂ*stuffÂ*off,Â*that isÂ*safeÂ*toÂ*moveÂ*(theÂ*bitsÂ*thatÂ*won'tÂ*affec tÂ*theÂ*current stateÂ*machine),Â*orÂ*youÂ*canÂ*duplicateÂ*theÂ*wh oleÂ*thingÂ*once inÂ*aÂ*while.Â*AndÂ*leaveÂ*theÂ*secondÂ*deviceÂ*di sconnected,Â*andÂ*sitting inÂ*theÂ*cornerÂ*forÂ*emergencies. I keep a 2nd copy on a 2nd drive of all my backups. I used to work in a work environment where we pulled the tape (yes tape drive, that's how long ago it was) out of the machine in the morning and placed it in one of the secretaries purses to take home. Always one off site. Al |
#6
|
|||
|
|||
Macrium scheduling
Big Al wrote:
On 11/19/19 8:50 AM, Paul wrote: You*can*either*move*the*stuff*off,*that is*safe*to*move*(the*bits*that*won't*affect*the*cu rrent state*machine),*or*you*can*duplicate*the*whole*thi ng*once in*a*while.*And*leave*the*second*device*disconnect ed,*and*sitting in*the*corner*for*emergencies. I keep a 2nd copy on a 2nd drive of all my backups. I used to work in a work environment where we pulled the tape (yes tape drive, that's how long ago it was) out of the machine in the morning and placed it in one of the secretaries purses to take home. Always one off site. Al Thanks Al. I'm tempted to do the same, largely because I can understand what's going on more easily! But I'm going to have another coffee and read Paul's comprehensive response at least once more before I decide. P.S. The tapes I can remember from my training days with IBM would need *very* large purses ;-) Terry, East Grinstead, UK |
#7
|
|||
|
|||
Macrium scheduling
Paul wrote:
Terry Pinnell wrote: Some background before I get to my specific question. I setup Macrium Reflect (Free) over three years ago after helpful advice here. After a few months of occasional checks I left it to do its stuff and essentially forgot about it. It was scheduled very early every Saturday morning so I didn't often get to see it start anyway. But yesterday I discovered that every weekly run since May had been unsuccessful ;-( (The reason turned out to be that somehow the drive letter of my destination external USB drive letters had been altered.) -------------------- In the course of reconfiguring I'm therefore reconsidering my choice of various scheduling templates, I think my previous was 'Incremental Backup Set', and that now seems perhaps a bit over complicated for a solo end user with no network stuff? I'd appreciate others' opinions or experience please. METHODS #1 Grandfather, Father, Son: Daily Incremental ("Son"), weekly Differential ("Father"), and monthly Full ("Grandfather") backups. #2 Differential Backup Set: A Full backup is created periodically followed by daily Differential backups. #3 Incremental Backup Set: A Full backup is created periodically followed by daily Incremental backups. #4 Incrementals Forever: Incrementals forever optimizes backup space and time by only ever creating a single Full backup. After this Incremental backups are created ad infinitum. The Full backup is consolidated with subsequent Incremental backups once the specified number of Incremental backups is reached. This is also known as a Synthetic Full backup. -------------------- As well as the 'method', any tips on optimal sequence of the full and incremental runs? Oh, and how often do others make a new recovery USB? Terry, East Grinstead, UK Since Uwe was along today, in his honor we can promote this program. It's a Drive Letter Manager to help with USB devices. https://www.uwe-sieber.de/usbdlm_e.html ******* #2 and #3 are part of implementing #1. #4 is a fancy version of #3. So that leaves a discussion of #1. #1 starts with a Full, followed by incrementals. You can have as many incrementals as you want, but they have to remain readable. If you have 200 of them, and the third one croaks, you have a large pile of unusable incrementals and a very poor outcome. The incremental chain must be truncated, by doing a Full every once in a while. Then the incrementals start again, being applied to that Full. This is a reliability issue. If you keep 200 Incrementals with respect to a Full, all 200 have to be readable when you "restore to the 200th one". Macrium (as it should), treats a Full and a set of Incrementals as a "complete set". If you need to free up space, it generally deletes a Full plus the associated Incrementals, as a set. So what does the "Father" do ? since it's a differential, it works out the differences since the original Full. It allows the incrementals to be deleted, and more incrementals to be run. At the end of the week, another Differential is created, and the incrementals erased (since the Differential captured everything they did, and now the Incrementals are redundant). The Differentials are points in time, so if we need to go back a week, we can. The incrementals give us one day resolution (but only for the last week). While the differentials allow us to go back a week at a time only. When the end of the month arrives, a Full is made. This allows the previous Full and the Differentials to be erased if space is needed. The Fulls collected each month, allow us to go back a month for each one. But they're likely to be large files, so keeping 12 of those would be a handful. At some point, the Capacity Manager thing is going to start throwing month fulls, overboard. If I did that sequence on this machine, my backup drive would be exhausted by January with that pattern. The incrementals and weekly differentials wouldn't be that big of a burden, but collecting Fulls once a month will add up. (Twelve times 2TB would be 24TB per year.) The Incrementals Forever idea, is to have the efficiency of Incrementals (really short backup interval), with the capping off using a synthetic full, for some period of time. But by itself, #4 isn't a complete strategy, whereas #1 is closer to a complete strategy (a strategy the IT department might use, with some adjustments). Once the monthly Fulls are made by #1, you have the option of moving them off the drive to make room for the pattern. For example, since my backup drive could not handle too many Fulls collected once a month, when I hit January, I might move November to another drive. Leaving December and January on there. If I were to delete the current Full, then all the associated Incrementals and Differentials should disappear at the same instant. Whereas if I throw away the July Full, it is independent of what is happening in November. If you're going to adopt #1, I'd probably want at least room for two Fulls, and then move the excess (old) Fulls somewhere else. Once you have 24 Fulls in hand, you could keep the oldest one and the twelth one, and those are your yearly backups. Then keep monthly ones that the sequence is generating. For example, I once had to go back two years, to discover when my drive got a certain "unremovable driver" added to it. So old backups come in handy, "once in a lifetime". If I only kept those with a temporal resolution of "once a year", I probably wouldn't be that upset. So while #1 is the correct sequence, it hasn't made allowance for a yearly plan as such. You can do that manually of course. You can add more "tiers" to the plan, and make it a bit more sensible with respect to the size of the backup drive. For example, I don't really need monthly ones forever, and after three months, I could keep ones from every quarter and from every year after that. I would need some kind of RAID array, to make a pool big enough for #1 without modification :-) If you had a dinky C: and a huge external, then perhaps #1 could run for quite a while without intervention. Paul Great post, thanks Paul. It should get a sticky in the Macrium forum if it doesn't have one already. As per my reply to Al, simplicity weighs heavily in my thinking. Particularly if that unsought day arises and I have to restore an image. Can I really be confident of navigating safely to what seems a rather complex 'set' of G-F-S files and faithfully restoring them? If I'd already done it once then probably Yes, but... And I also seriously doubt I'd ever want to restore a state more than a month or three ago, given the frequent changes I make, plus WU's. While not in your terabyte league, capacity isn't a trivial factor here; last image was over 150 GB. I'll sleep on it for a few nights. -------------------- I was about to implement the automatic failure email feature, but see it needs an upgrade. I'll probably do that, but meanwhile maybe write a simple macro. It would just open Macrium soon after each run, click the Log tab, expand the top entry, check for a red cross, and display a prominent message. -------------------- Will also look at that handy drive lettering tool you suggested. Terry, East Grinstead, UK |
#8
|
|||
|
|||
Macrium scheduling
Terry Pinnell wrote:
Some background before I get to my specific question. I setup Macrium Reflect (Free) over three years ago after helpful advice here. After a few months of occasional checks I left it to do its stuff and essentially forgot about it. It was scheduled very early every Saturday morning so I didn't often get to see it start anyway. But yesterday I discovered that every weekly run since May had been unsuccessful ;-( (The reason turned out to be that somehow the drive letter of my destination external USB drive letters had been altered.) -------------------- In the course of reconfiguring I'm therefore reconsidering my choice of various scheduling templates, I think my previous was 'Incremental Backup Set', and that now seems perhaps a bit over complicated for a solo end user with no network stuff? I'd appreciate others' opinions or experience please. METHODS #1 Grandfather, Father, Son: Daily Incremental ("Son"), weekly Differential ("Father"), and monthly Full ("Grandfather") backups. Reflect doesn't do true GFS backups where, for example, a grandfather father and son scheduled at the same time would only have the grandfather backup run. I haven't found a consumer-grade backup program that does a true GFS. Instead they just use the GFS concept for scheduling. If you schedule them at the same time, you don't which will run. If you schedule the grandfather with the father starting 5 minutes later and the son starting 10 minutes later, you know the order they will run: grandfather, father, and then son. Reflect will only run one backup at a time, so the father pends until the grandfather finishes, and the son pends until the father finishes. That I did was to schedule the grandfather backup (full) to run on the first Saturday of the month, the father backup (differential) on Sunday (at the same time but different day as the grandfather), and the son backup (incremental) to start 5 minutes later on every day. I could have grandfather run on the first Saturday, father on the first Saturday but 5 minutes later, and the son on every day 10 minutes later, too. Since the grandfather just ran, the father would be very small. Not much changed between when the grandfather and father backups ran, and the father backup is mostly the size of a holding file but will have little inside of it. The son will be even more tiny since it ran after the grandfather and father backups. The first Saturday will be at the end of the first week of every month. If you choose first anyOtherDayOfTheWeek, that might not be until the 2nd week of the month. If you choose a day of the month, that varies all over the place as to which day-of-the-week it falls on. #2 Differential Backup Set: A Full backup is created periodically followed by daily Differential backups. Which is what you get in the free version of Reflect (incrementals are absent in the free version). If you run Differentials every day, they are effectively incrementals but based off the full backup. Don't know how long or at what interval is "periodically". The longer the chain of differentials, the more fragile it becomes. Should any differential backup file become corrupt or go missing, all differentials from that corrupted/missing differential will be unusable. #3 Incremental Backup Set: A Full backup is created periodically followed by daily Incremental backups. The problem there is the chain will get longer. You don't want to keep running full backups at short intervals because of their size. You will chew up your storage space rather quickly with lots of full backups. If you run a full backup once per month, that means the chain of 30 incrementals will have 30 links in the chain (plus the full backup). If you run them over a week, the chain will be shorter, so less loss if the chain breaks, but you'll be running 4 times as many full backups with them running every weekend. Whether that is doable depends on how large is the backup storage media and how big is the source that is getting backed up. #4 Incrementals Forever: Incrementals forever optimizes backup space and time by only ever creating a single Full backup. After this Incremental backups are created ad infinitum. The Full backup is consolidated with subsequent Incremental backups once the specified number of Incremental backups is reached. This is also known as a Synthetic Full backup. That does one full backup followed thereafter with incrementals. The chain keeps getting longer and longer. However, if you shorten the retention interval, the incrementals that are older than that will get rolled into the full backup (which means you are touching the full backup, the primary one, instead of leaving it alone for use only during a restore). Retouching the base (full) backup repeatedly is a recipe for disaster. The idea is to keep the chain shorter by limiting how many incrementals to reduce the chain length, but that is at the risk of touching the full backup. The other problem is that restoring the full backup (so you can then use the incrementals in a restore) is that you will be restoring EVERYTHING ever changed on your computer. Something you installed but later uninstalled, some registry tweak you changed but unchanged, and anything changed and unchanged will be recorded in both states in the oversized full backup file. Rolling in incrementals doesn't eliminate any entries in the full backup that would be voided by later changes in the incrementals. The full backup that keeps getting changed (instead of keeping it safely quiescent) will keep getting larger (unless you make no changes at all). Oh, and how often do others make a new recovery USB? When there is a major version change to Reflect. The PE doesn't change what it uses (since you keep using the same version that is compatible with the installed version of Windows), but features or fixes will change in a new version which could apply to using Reflect under the PE. For the boot-time image (to load Reflect on booting the computer which means storing the boot image on an internal drive), and when there is a sub-minor version change (e.g., 7.1 to 7.2), I remove Reflect's boot-time option, and reinstall it (which builds a new boot image and stores on the internal drive). That is pretty easy. Just go into the menu, remove the boot option, re-enable the boot option. The Windows PE under which Reflect will run when booted won't change. You still use the same WinPE as before that is compatible with your installed version of Windows. For the bootable optical disc (CD), I don't bother burning a new one until there is a major change to Reflect (e.g., version 6.x to 7.x). When I restore, I'm highly likely to use the boot-time image: boot the computer, choose with OS image to load, very easy. The CD is a backup should the drive get corrupted, fried, died, or replaced where are the boot images. Which bring up another point: If the storage media goes bad where you saved your backup files, how are you going to restore? If the drive goes bad or is lost, you no longer have any of your backups. I have an internal HDD (be a waste to use an SDD) to which I save my backups. You did not say which *edition* of Reflect that you have. I have the Home edition, and that does not have the option to copy the finished backup to another location, like to another drive (which could be a USB drive). It doesn't even have an option to copy the backup file via FTP to another host. I'd have to pay more to get the auto-copy and FTP options. Instead I used to use the free SyncBack program, but have since moved up to the SyncBack Lite version (because it could copy inuse/locked files using the Volume Shadow Copy feature in Windows). That wasn't needed for Reflect, but for other file-sync jobs that I wanted SyncBack to do. The Lite version has since been retired, and they replaced it with SyncBack SE (which, as I recall, is $10 more than Lite's price). I never bothered to leave the Lite version, because they still support it with updates, so I don't know if the $10 gives me anything more than what the Lite version gives me for features (and those I'd use). I use SyncBack (even free version works) to copy the backup files from the internal SATA HDD to a USB HDD. That gives me a copy of the backup files. If the USB HDD fails, I get another. If the internal SATA HDD fails, use the USB HDD in the interim for any restores, and replace the internal SATA HDD to copy the backups from the USB HDD (since restores are much faster using the internal SATA HDD, and the USB HDD is really just a backup of my backups). After Reflects ends a backup job, the backup file won't be inuse or locked, so even the free version of SyncBack will work to copy my backups files from the internal HDD to the USB HDD. Alas, Reflect doesn't have easy-to-configure pre- and post-backup job definitions. It has no means of defining to run a script (batch file) before or after its backup. It does let you define, for example, MS-DOS batch files that will run the backup, and you can edit them to add commands before and after the command to run the backup. Alternatively, Reflect uses the Windows Task Scheduler to define its scheduled backups, so you could copy the command line for its scheduled event, put it in a batch file, edit it to add pre- and post-backup commands, and create a new scheduled event in Task Scheduler (and disable/delete the one that Refect created). I've run into problems doing either method which has both use a .bat script that can be edited when used on a Windows 10 host. When the batch files run, Reflect knows how to elevate itself to admin privileges, but other commands you run in the batch file won't be elevated which means they may fail. Yes, you can edit the Task Scheduler event to run with admin privileges (escalated), but that won't work if they run under a Microsoft account instead of a local/offline account (where both have admin privileges). There are reasons why I want the backups to run under my account, and reasons why I still use a Microsoft account instead of a local/offline account. Task Scheduler in Windows 10 only knows how to handle admin-level scheduled events that use a local/offline account. For Microsoft accounts, Task Scheduler doesn't know how to access the login credentials for that account. That means you cannot run events as admin, because Task Scheduler requires it knows the login credentials for an admin account, but Task Scheduler cannot get the login credentials for a Microsoft account. Yeah, Microsoft ****ed up in Task Scheduler to run events under a Microsoft account even when it has admin privileges. There are few (not many, but a couple) advantages to using a Microsoft account. I could run the events in Task Scheduler under a different account, but that has problems (too long to get into this already very long reply). I found a trick of defining an event in Task Scheduler that triggers on an event, and when Reflect finishes a backup an event gets recorded (you can see it in the Event Viewer). I first have SyncBack create its scheduled sync jobs in Task Scheduler. In Task Scheduler, I edit those events to change the run trigger from specifying a time to run "On an event". The "scheduled" events for SyncBack would fire when the Reflect backups finished. The definition of the event trigger is difficult to understand, and you have to read the event to get the ID, and the custom event script is hard to decipher, so it's tricky and I won't get into it here. If you're not using a Microsoft account under which you run your Reflect backups (you're using a local aka offline account with admin privs), you can use Reflect's MS-DOS Batch Files tab to let you edit .bat files to add commands before and after the backup starts/ends, like running SyncBack after the backup to save a copy of the backup file to another drive or even another host (SyncBack will do file sync to local drives whether internal or USB attached, to network mapped drives, or to other hosts via FTP). As for the USB drive letters changing, not a problem when the backup files get saved to an internal HDD where drive don't change unless you commit surgery on your disk setup. Reflect Home doesn't have an option to target a USB drive other than by a drive letter. SyncBack (well, in the Lite version, don't have the free version to check) lets you specify the target drive by drive letter, serial number, or label. For example, my USB-attached HDD is drive E:, serial 6E89-9DA2, or label WDC2TB_USBHD (for a Western Digital 2TB USB HD). I give my each of USB drives a unique label, so I could use that to find them no matter which drive letter they happen to get. Using the serial number would get confusing to figure out later which I had specified a long time ago. Even if the drive letter changed, SyncBack can still find the USB HDD. A feature available in SyncBack but not in Reflect. Alternatively, there are tools that will try to keep the drive letter assignment the same between ejects and injects of a USB drive, like UWE-Sieber's USBDLM (USB Drive Letter Manager) at: https://www.uwe-sieber.de/usbdlm_e.html Obviously if, say, drive letter F: has already been assigned to a drive then plugging in the USB drive won't let it get F: again. You have to use USBDLM to define which drive letter gets assigned to which USB device for all of them you may use, so each gets a unique one to eliminate the conflict in the first place. Lots of prep work, plus you may discard and get new USB drives and have to update USBDLM again. Normally Windows should not change the drive letter of a USB drive unless the old drive letter it had before is currently in use. I do remember having to sometimes go into Disk Management (diskmgmt.msc) to assign a drive letter there for it to get remembered then the USB drive got plugged in later. I don't remember the circumstances, but assigning the drive letter in Disk Management was better at making sticky the drive letter assignment (as long as there was no conflict in drive letter assignments). Still awake? |
#9
|
|||
|
|||
Macrium scheduling
VanguardLH wrote:
Terry Pinnell wrote: Some background before I get to my specific question. I setup Macrium Reflect (Free) over three years ago after helpful advice here. After a few months of occasional checks I left it to do its stuff and essentially forgot about it. It was scheduled very early every Saturday morning so I didn't often get to see it start anyway. But yesterday I discovered that every weekly run since May had been unsuccessful ;-( (The reason turned out to be that somehow the drive letter of my destination external USB drive letters had been altered.) -------------------- In the course of reconfiguring I'm therefore reconsidering my choice of various scheduling templates, I think my previous was 'Incremental Backup Set', and that now seems perhaps a bit over complicated for a solo end user with no network stuff? I'd appreciate others' opinions or experience please. METHODS #1 Grandfather, Father, Son: Daily Incremental ("Son"), weekly Differential ("Father"), and monthly Full ("Grandfather") backups. Reflect doesn't do true GFS backups where, for example, a grandfather father and son scheduled at the same time would only have the grandfather backup run. I haven't found a consumer-grade backup program that does a true GFS. Instead they just use the GFS concept for scheduling. If you schedule them at the same time, you don't which will run. If you schedule the grandfather with the father starting 5 minutes later and the son starting 10 minutes later, you know the order they will run: grandfather, father, and then son. Reflect will only run one backup at a time, so the father pends until the grandfather finishes, and the son pends until the father finishes. That I did was to schedule the grandfather backup (full) to run on the first Saturday of the month, the father backup (differential) on Sunday (at the same time but different day as the grandfather), and the son backup (incremental) to start 5 minutes later on every day. I could have grandfather run on the first Saturday, father on the first Saturday but 5 minutes later, and the son on every day 10 minutes later, too. Since the grandfather just ran, the father would be very small. Not much changed between when the grandfather and father backups ran, and the father backup is mostly the size of a holding file but will have little inside of it. The son will be even more tiny since it ran after the grandfather and father backups. The first Saturday will be at the end of the first week of every month. If you choose first anyOtherDayOfTheWeek, that might not be until the 2nd week of the month. If you choose a day of the month, that varies all over the place as to which day-of-the-week it falls on. #2 Differential Backup Set: A Full backup is created periodically followed by daily Differential backups. Which is what you get in the free version of Reflect (incrementals are absent in the free version). If you run Differentials every day, they are effectively incrementals but based off the full backup. Don't know how long or at what interval is "periodically". The longer the chain of differentials, the more fragile it becomes. Should any differential backup file become corrupt or go missing, all differentials from that corrupted/missing differential will be unusable. #3 Incremental Backup Set: A Full backup is created periodically followed by daily Incremental backups. The problem there is the chain will get longer. You don't want to keep running full backups at short intervals because of their size. You will chew up your storage space rather quickly with lots of full backups. If you run a full backup once per month, that means the chain of 30 incrementals will have 30 links in the chain (plus the full backup). If you run them over a week, the chain will be shorter, so less loss if the chain breaks, but you'll be running 4 times as many full backups with them running every weekend. Whether that is doable depends on how large is the backup storage media and how big is the source that is getting backed up. #4 Incrementals Forever: Incrementals forever optimizes backup space and time by only ever creating a single Full backup. After this Incremental backups are created ad infinitum. The Full backup is consolidated with subsequent Incremental backups once the specified number of Incremental backups is reached. This is also known as a Synthetic Full backup. That does one full backup followed thereafter with incrementals. The chain keeps getting longer and longer. However, if you shorten the retention interval, the incrementals that are older than that will get rolled into the full backup (which means you are touching the full backup, the primary one, instead of leaving it alone for use only during a restore). Retouching the base (full) backup repeatedly is a recipe for disaster. The idea is to keep the chain shorter by limiting how many incrementals to reduce the chain length, but that is at the risk of touching the full backup. The other problem is that restoring the full backup (so you can then use the incrementals in a restore) is that you will be restoring EVERYTHING ever changed on your computer. Something you installed but later uninstalled, some registry tweak you changed but unchanged, and anything changed and unchanged will be recorded in both states in the oversized full backup file. Rolling in incrementals doesn't eliminate any entries in the full backup that would be voided by later changes in the incrementals. The full backup that keeps getting changed (instead of keeping it safely quiescent) will keep getting larger (unless you make no changes at all). Oh, and how often do others make a new recovery USB? When there is a major version change to Reflect. The PE doesn't change what it uses (since you keep using the same version that is compatible with the installed version of Windows), but features or fixes will change in a new version which could apply to using Reflect under the PE. For the boot-time image (to load Reflect on booting the computer which means storing the boot image on an internal drive), and when there is a sub-minor version change (e.g., 7.1 to 7.2), I remove Reflect's boot-time option, and reinstall it (which builds a new boot image and stores on the internal drive). That is pretty easy. Just go into the menu, remove the boot option, re-enable the boot option. The Windows PE under which Reflect will run when booted won't change. You still use the same WinPE as before that is compatible with your installed version of Windows. For the bootable optical disc (CD), I don't bother burning a new one until there is a major change to Reflect (e.g., version 6.x to 7.x). When I restore, I'm highly likely to use the boot-time image: boot the computer, choose with OS image to load, very easy. The CD is a backup should the drive get corrupted, fried, died, or replaced where are the boot images. Which bring up another point: If the storage media goes bad where you saved your backup files, how are you going to restore? If the drive goes bad or is lost, you no longer have any of your backups. I have an internal HDD (be a waste to use an SDD) to which I save my backups. You did not say which *edition* of Reflect that you have. I have the Home edition, and that does not have the option to copy the finished backup to another location, like to another drive (which could be a USB drive). It doesn't even have an option to copy the backup file via FTP to another host. I'd have to pay more to get the auto-copy and FTP options. Instead I used to use the free SyncBack program, but have since moved up to the SyncBack Lite version (because it could copy inuse/locked files using the Volume Shadow Copy feature in Windows). That wasn't needed for Reflect, but for other file-sync jobs that I wanted SyncBack to do. The Lite version has since been retired, and they replaced it with SyncBack SE (which, as I recall, is $10 more than Lite's price). I never bothered to leave the Lite version, because they still support it with updates, so I don't know if the $10 gives me anything more than what the Lite version gives me for features (and those I'd use). I use SyncBack (even free version works) to copy the backup files from the internal SATA HDD to a USB HDD. That gives me a copy of the backup files. If the USB HDD fails, I get another. If the internal SATA HDD fails, use the USB HDD in the interim for any restores, and replace the internal SATA HDD to copy the backups from the USB HDD (since restores are much faster using the internal SATA HDD, and the USB HDD is really just a backup of my backups). After Reflects ends a backup job, the backup file won't be inuse or locked, so even the free version of SyncBack will work to copy my backups files from the internal HDD to the USB HDD. Alas, Reflect doesn't have easy-to-configure pre- and post-backup job definitions. It has no means of defining to run a script (batch file) before or after its backup. It does let you define, for example, MS-DOS batch files that will run the backup, and you can edit them to add commands before and after the command to run the backup. Alternatively, Reflect uses the Windows Task Scheduler to define its scheduled backups, so you could copy the command line for its scheduled event, put it in a batch file, edit it to add pre- and post-backup commands, and create a new scheduled event in Task Scheduler (and disable/delete the one that Refect created). I've run into problems doing either method which has both use a .bat script that can be edited when used on a Windows 10 host. When the batch files run, Reflect knows how to elevate itself to admin privileges, but other commands you run in the batch file won't be elevated which means they may fail. Yes, you can edit the Task Scheduler event to run with admin privileges (escalated), but that won't work if they run under a Microsoft account instead of a local/offline account (where both have admin privileges). There are reasons why I want the backups to run under my account, and reasons why I still use a Microsoft account instead of a local/offline account. Task Scheduler in Windows 10 only knows how to handle admin-level scheduled events that use a local/offline account. For Microsoft accounts, Task Scheduler doesn't know how to access the login credentials for that account. That means you cannot run events as admin, because Task Scheduler requires it knows the login credentials for an admin account, but Task Scheduler cannot get the login credentials for a Microsoft account. Yeah, Microsoft ****ed up in Task Scheduler to run events under a Microsoft account even when it has admin privileges. There are few (not many, but a couple) advantages to using a Microsoft account. I could run the events in Task Scheduler under a different account, but that has problems (too long to get into this already very long reply). I found a trick of defining an event in Task Scheduler that triggers on an event, and when Reflect finishes a backup an event gets recorded (you can see it in the Event Viewer). I first have SyncBack create its scheduled sync jobs in Task Scheduler. In Task Scheduler, I edit those events to change the run trigger from specifying a time to run "On an event". The "scheduled" events for SyncBack would fire when the Reflect backups finished. The definition of the event trigger is difficult to understand, and you have to read the event to get the ID, and the custom event script is hard to decipher, so it's tricky and I won't get into it here. If you're not using a Microsoft account under which you run your Reflect backups (you're using a local aka offline account with admin privs), you can use Reflect's MS-DOS Batch Files tab to let you edit .bat files to add commands before and after the backup starts/ends, like running SyncBack after the backup to save a copy of the backup file to another drive or even another host (SyncBack will do file sync to local drives whether internal or USB attached, to network mapped drives, or to other hosts via FTP). As for the USB drive letters changing, not a problem when the backup files get saved to an internal HDD where drive don't change unless you commit surgery on your disk setup. Reflect Home doesn't have an option to target a USB drive other than by a drive letter. SyncBack (well, in the Lite version, don't have the free version to check) lets you specify the target drive by drive letter, serial number, or label. For example, my USB-attached HDD is drive E:, serial 6E89-9DA2, or label WDC2TB_USBHD (for a Western Digital 2TB USB HD). I give my each of USB drives a unique label, so I could use that to find them no matter which drive letter they happen to get. Using the serial number would get confusing to figure out later which I had specified a long time ago. Even if the drive letter changed, SyncBack can still find the USB HDD. A feature available in SyncBack but not in Reflect. Alternatively, there are tools that will try to keep the drive letter assignment the same between ejects and injects of a USB drive, like UWE-Sieber's USBDLM (USB Drive Letter Manager) at: https://www.uwe-sieber.de/usbdlm_e.html Obviously if, say, drive letter F: has already been assigned to a drive then plugging in the USB drive won't let it get F: again. You have to use USBDLM to define which drive letter gets assigned to which USB device for all of them you may use, so each gets a unique one to eliminate the conflict in the first place. Lots of prep work, plus you may discard and get new USB drives and have to update USBDLM again. Normally Windows should not change the drive letter of a USB drive unless the old drive letter it had before is currently in use. I do remember having to sometimes go into Disk Management (diskmgmt.msc) to assign a drive letter there for it to get remembered then the USB drive got plugged in later. I don't remember the circumstances, but assigning the drive letter in Disk Management was better at making sticky the drive letter assignment (as long as there was no conflict in drive letter assignments). Still awake? Many thanks. Yes, still awake, but will re-read more carefully tomorrow! This is Macrium Reflect 7.2 (Free), 64-bit. Terry, East Grinstead, UK |
#10
|
|||
|
|||
Macrium scheduling
Terry Pinnell wrote:
Wow, that was an impressively thorough explanation. Collectively with Paul's this thread is now becomes 'Macrium Scheduling Tutorial'! I was confused as to which of those four 'methods' I listed in my opening post I've currently specified. Instead of a link I was trying to be helpful in pasting, but should have avoided adding my own (wrong) numbers! The XML indicates that I'm using 'method 3', which I now see is the fourth option, not the third. And I gather 'Auto' is 'Incrementals Forever' or 'Synthetic Full Backup'. Here's my current schedule, specified yesterday: https://www.dropbox.com/s/7mawbl3rvc...le-1.jpg?raw=1 Bottom line: I reckon that with simplicity as a key motivator for me, and no capacity issue, I'm favouring just Fulls, weekly. Terry, East Grinstead, UK As long as the ratio of a Full, to the size of the backup device, leaves room for lots of backups, then your situation will be "mostly maintenance free". You'll only have cleanup or migration to do "once in a while". It's when there is a danger of running out of space, that more cunning is required. I would likely have to buy a larger external for myself, if three fulls couldn't fit on the external drive. That would be my tolerance for pain. Any amount of space less than that, I'd just do them manually, with no schedule. (Which is what I do here anyway.) When my Internet broke about a week ago, I passed the time by doing a Full :-) That's my "schedule". The only drive that would make a decent external for me, would be the new 100TB SSD they came out with. (No price was provided, and it just looked like a mockup, rather than the real thing. Form factor was 3.5".) That should be more expensive than a small car. Then, just about any schedule I used, would work. Paul |
#11
|
|||
|
|||
Macrium scheduling
Terry Pinnell wrote:
Bottom line: I reckon that with simplicity as a key motivator for me, and no capacity issue, I'm favouring just Fulls, weekly. I like to do a scheduled backup every day in the early morning (while I'm sleeping). I don't know when Microsoft might push yet another update. Some can be delayed, like the 1909 update, but many get shoved onto your computer awaiting the next reboot. With Microsoft's history of pushing bad updates (some causing castastrophies), I want a backup the morning they push an update, so I have a decent escape route (Microsoft's system restore or update backouts are not reliable). I might also flub something in Windows or some program, and need to restore. I don't want to lose everything for a whole week. I might've just completed some legal doc that I'm still gathering info to complete, and need to print and send later. Quite often finding the correct doc(s) the gov't wants is a big effort. For some, that's why they hire lawyers to do all that. The gov't loves paperwork, but it has to just the right paperwork. The granularity of your backups depends on how much you're willing to lose should you have to restore from malware, bad updates, a bad tweak, software installs that are nasty to remove, or failed hardware. To me, a week is way to long. For others, they may go with a monthly backup. You could keep using the weekly backups, and watch how many restores you do that result in too much or critical data or config loss. I started them monthly. Bang, had to restore, way too much lost. Moved to weeklies. Bang, had to restore, too much lost. Moved to daily. Since they are scheduled, it's not like I get nuisanced to run them. I just have to monitor my backup drives to make sure they're not running out of room for new backups (which means I've had to tailor the retentions to automatically free up space). |
#12
|
|||
|
|||
Macrium scheduling
Paul wrote:
Terry Pinnell wrote: Wow, that was an impressively thorough explanation. Collectively with Paul's this thread is now becomes 'Macrium Scheduling Tutorial'! I was confused as to which of those four 'methods' I listed in my opening post I've currently specified. Instead of a link I was trying to be helpful in pasting, but should have avoided adding my own (wrong) numbers! The XML indicates that I'm using 'method 3', which I now see is the fourth option, not the third. And I gather 'Auto' is 'Incrementals Forever' or 'Synthetic Full Backup'. Here's my current schedule, specified yesterday: https://www.dropbox.com/s/7mawbl3rvc...le-1.jpg?raw=1 Bottom line: I reckon that with simplicity as a key motivator for me, and no capacity issue, I'm favouring just Fulls, weekly. Terry, East Grinstead, UK As long as the ratio of a Full, to the size of the backup device, leaves room for lots of backups, then your situation will be "mostly maintenance free". You'll only have cleanup or migration to do "once in a while". It's when there is a danger of running out of space, that more cunning is required. I would likely have to buy a larger external for myself, if three fulls couldn't fit on the external drive. That would be my tolerance for pain. Any amount of space less than that, I'd just do them manually, with no schedule. (Which is what I do here anyway.) When my Internet broke about a week ago, I passed the time by doing a Full :-) That's my "schedule". The only drive that would make a decent external for me, would be the new 100TB SSD they came out with. (No price was provided, and it just looked like a mockup, rather than the real thing. Form factor was 3.5".) That should be more expensive than a small car. Then, just about any schedule I used, would work. Paul Can you say why your images are so gigantic? A couple of others I've communicated with regard my 180 GB (imaging to about 135 GB) as being very large compared to their 30-50 GB. Is it because you're imaging almost your entire PC contents? ** When I set up this PC in 2016 I didn't feel technically confident enough about partitioning to split the OS and the rest on my 256 GB SSD, C. (Still don't really.) And I felt more comfortable having all my several hundred programs on it as well as the OS, in the hope that would make for easier recovery. And I have data there too, mostly those files associated with my video-making which need especially fast access. (The great bulk of my data is on my 4 TB internal HD, D.) But if I had split off the OS then I could now simply use the Macrium option "Create an image of the partitions required to back up and restore Windows", which would be much smaller. ** After taking advantage of the Black Friday 50% discount yesterday I upgraded. Making my first Full I was dismayed to see an estimated 7 hours for completion and something like 1.3 TB. (Previous Fulls had taken about 30 mins.) Turned out I had clicked 'Image selected disks on this computer'; only two of the small boxes against my 7 drives were checkmarked and I'd hastily assumed that was another neat way of starting an image. (I'm still not exactly clear how those boxes are meant to be used.) After promptly canceling and then reverting to the correct method of clicking 'Image this disk' below C in the main window, it made the image (on my 3 TB external USB drive I) in about 30 mins. BTW, I'm backing up all non-OS files nightly and weekly, which is why I'm confining imaging to C. Terry, East Grinstead, UK |
#13
|
|||
|
|||
Macrium scheduling
VanguardLH wrote:
Terry Pinnell wrote: Bottom line: I reckon that with simplicity as a key motivator for me, and no capacity issue, I'm favouring just Fulls, weekly. I like to do a scheduled backup every day in the early morning (while I'm sleeping). I don't know when Microsoft might push yet another update. Some can be delayed, like the 1909 update, but many get shoved onto your computer awaiting the next reboot. With Microsoft's history of pushing bad updates (some causing castastrophies), I want a backup the morning they push an update, so I have a decent escape route (Microsoft's system restore or update backouts are not reliable). Indeed. OT in this thread but... I may post again about my frequently vanishing SRs. I might also flub something in Windows or some program, and need to restore. I don't want to lose everything for a whole week. I might've just completed some legal doc that I'm still gathering info to complete, and need to print and send later. Quite often finding the correct doc(s) the gov't wants is a big effort. For some, that's why they hire lawyers to do all that. The gov't loves paperwork, but it has to [be] just the right paperwork. That's another chord that rings loud for me. OT again but... I have some stock from my employee share plan with IBM UK. For reasons I've never fathomed it's managed in the USA, not UK. I've just spent several *weeks* (translantic calls, emails and surface mail) meeting the IRS's need for a form called W-8BEN, completed to rigid perfection. The granularity of your backups depends on how much you're willing to lose should you have to restore from malware, bad updates, a bad tweak, software installs that are nasty to remove, or failed hardware. To me, a week is way to long. For others, they may go with a monthly backup. You could keep using the weekly backups, and watch how many restores you do that result in too much or critical data or config loss. I started them monthly. Bang, had to restore, way too much lost. Moved to weeklies. Bang, had to restore, too much lost. Moved to daily. Since they are scheduled, it's not like I get nuisanced to run them. I just have to monitor my backup drives to make sure they're not running out of room for new backups (which means I've had to tailor the retentions to automatically free up space). You're running G-F-S, yes? How much capacity have you allocated for your images in total? Although ideally I too would prefer daily imaging, my leaning towards weekly is that my nightly and weekly backups (using SecondCopy for decades) would allow speedy recovery of data. Hardware disasters apart, of course. Anyway, I'll see how it goes and maybe change my mind. Terry, East Grinstead, UK |
#14
|
|||
|
|||
Macrium scheduling
Terry Pinnell wrote:
Paul wrote: Terry Pinnell wrote: Wow, that was an impressively thorough explanation. Collectively with Paul's this thread is now becomes 'Macrium Scheduling Tutorial'! I was confused as to which of those four 'methods' I listed in my opening post I've currently specified. Instead of a link I was trying to be helpful in pasting, but should have avoided adding my own (wrong) numbers! The XML indicates that I'm using 'method 3', which I now see is the fourth option, not the third. And I gather 'Auto' is 'Incrementals Forever' or 'Synthetic Full Backup'. Here's my current schedule, specified yesterday: https://www.dropbox.com/s/7mawbl3rvc...le-1.jpg?raw=1 Bottom line: I reckon that with simplicity as a key motivator for me, and no capacity issue, I'm favouring just Fulls, weekly. Terry, East Grinstead, UK As long as the ratio of a Full, to the size of the backup device, leaves room for lots of backups, then your situation will be "mostly maintenance free". You'll only have cleanup or migration to do "once in a while". It's when there is a danger of running out of space, that more cunning is required. I would likely have to buy a larger external for myself, if three fulls couldn't fit on the external drive. That would be my tolerance for pain. Any amount of space less than that, I'd just do them manually, with no schedule. (Which is what I do here anyway.) When my Internet broke about a week ago, I passed the time by doing a Full :-) That's my "schedule". The only drive that would make a decent external for me, would be the new 100TB SSD they came out with. (No price was provided, and it just looked like a mockup, rather than the real thing. Form factor was 3.5".) That should be more expensive than a small car. Then, just about any schedule I used, would work. Paul Can you say why your images are so gigantic? A couple of others I've communicated with regard my 180 GB (imaging to about 135 GB) as being very large compared to their 30-50 GB. Is it because you're imaging almost your entire PC contents? ** When I set up this PC in 2016 I didn't feel technically confident enough about partitioning to split the OS and the rest on my 256 GB SSD, C. (Still don't really.) And I felt more comfortable having all my several hundred programs on it as well as the OS, in the hope that would make for easier recovery. And I have data there too, mostly those files associated with my video-making which need especially fast access. (The great bulk of my data is on my 4 TB internal HD, D.) But if I had split off the OS then I could now simply use the Macrium option "Create an image of the partitions required to back up and restore Windows", which would be much smaller. ** After taking advantage of the Black Friday 50% discount yesterday I upgraded. Making my first Full I was dismayed to see an estimated 7 hours for completion and something like 1.3 TB. (Previous Fulls had taken about 30 mins.) Turned out I had clicked 'Image selected disks on this computer'; only two of the small boxes against my 7 drives were checkmarked and I'd hastily assumed that was another neat way of starting an image. (I'm still not exactly clear how those boxes are meant to be used.) After promptly canceling and then reverting to the correct method of clicking 'Image this disk' below C in the main window, it made the image (on my 3 TB external USB drive I) in about 30 mins. BTW, I'm backing up all non-OS files nightly and weekly, which is why I'm confining imaging to C. Terry, East Grinstead, UK I discovered there are controls on the left of the Macrium window, to back up the entire PC. Whereas I usually work on one drive at a time, on the right. "Image selected disks on this computer" === whole computer backup 2.13TB of files I have a number of VM images, and those add up after a while. Those are "OSes", stored on data partitions, and used in things like VPC2007 (WinXP only) or VirtualBox ("everywhere"). I have VMs even sitting on temporary disks. It's because I have no inventory system, that I can't clean this stuff up. I do actually have an Inventory folder, but it's out of date. It's time for another scan for that thing. It's an all day job to do that properly. ******* The speed of a Macrium backup, is constrained by a couple of things. One is, if you select lightweight compression, I think it only uses one core for that. Whereas PIGZ or 7ZIP know how to use multiple cores, and 7ZIP now even knows how to use multiple cores during an unzip operation. Macrium doesn't seem to be all that curious about the options available to them. In addition, Macrium computes an MD5 or SHA1 checksum for the image it creates. When you do a "Verify" on a .mrimg file later, that's what it checks (the checksum), to detect whether the file has been damaged since it was made. And I did detect a couple of bad images using Verify, so it works. Macrium cannot back up faster than it can compute (whatever it uses for the checksum. Even with ideal storage devices, depending on your CPU it probably can't break about 300MB/sec. So even if you had two NVMe drives, the backup flowing from one to the other, would have a ballpark 300MB/sec limit. The fastest checksum calc I've seen here, is a CRC32 done at 1500MB/sec or so (7ZIP, menu). Which still "holds back" a good quality NVMe from showing its stuff. Even if Macrium switched to CRC32 for a checksum (a bad idea!), there would still be transfer speed limits. ******* On external drives, make sure the USB enclosure and the USB port, are real actual USB3 items. Blue colored connectors and so on. Bench the drive with HDTune and verify the read transfer rate seems reasonable for the enclosure type. I find sometimes, that the Windows disk caches misbehave a bit, and that results in some weird transfer rate fluctuations. When I see this, it makes me really angry :-) We pay good money for these OSes, to have this crap fixed and working properly. Paul |
#15
|
|||
|
|||
Macrium scheduling
Terry Pinnell wrote:
You're running G-F-S, yes? How much capacity have you allocated for your images in total? Reflect images my C: drive (OS + apps + data). I move data to drive D: that I don't want in the backups occupying disk space, because it is replaceable or a copy stored elsewhere. C: is a 1TB NVMe SSD, but only 103GB is currently used. My GFS is monthly full, weekly differential, and daily incrementals. Retention is: 26 weeks for fulls (monthly), 5 weeks for differentials (weekly), and 14 days for incrementals (dailies). The internal D: drive 2TB HDD stores the backup images, program exported configs/backups, and some rather large data files that don't need to be in the backups (and why they're on the same drive as the backups which you wouldn't do if they were in the backups). For example, a couple versions of WSUSoffline are on D: to save all the MS updates for Win7, Office 2016/2019, and Win10. C: SSD drive: 103GB used (of 1TB) D: HDD drive: 546GB used (of 2TB) 410GB for Reflect image backups (*) 13GB for wsusoffline (only have 1 copy now) 100GB for some large files E: USB drive: 619GB used (of 2TB) 410GB for copies of Reflect image backups 100GB for copies of some of the large files 9GB for a copy of C:\Downloads (**) (*) Includes a 70GB folder for Win7 full image taken before wiping and a fresh install of Win10, just in case I need data from that old setup. (**) Not the default Downloads folder in Win10 (which is under my account profile folder and used only for temporary downloads). C:\DOWNLOADS is where I store all my downloads for all programs, copied to E: (USB) and occasionally copied to a 64GB USB flash drive. For my Win10 desktop, image backups only go back to Aug 2019, because that's when I built the new box. 6 months retention means the old ones will start rolling out around Jan-Feb 2020. For now, I should have plenty of free space to accomodate the later backups (along with the other files I'm saving on the "backup" drives). If I went to a year's retention, I'd be looking to replace the 2TB drives with 4TB drives, or larger, depending on my budget at the time. Or, if I consumed a lot more space on the 1TB NVMe SSD, I'd have to get bigger backup drives. Note: Not all the image backup files for Reflect are for the scheduled backup jobs. Occasionally I create an image backup before some change, like an incremental if I do MS updates in the evening (since there wouldv'e been an incremental that day early in the morning), or a differential or full depending on the magnitude of the change for which I want an escape route for recovery. So, besides the scheduled backups, there are a few manual backups saved on D: (and then sync'ed to E: for a 2nd copy of the backups). My old boxes go to family to upgrade their hardware. So, as yet, I haven't had a 2nd box at home to use to save backup files via FTP. That would be safer for the backup copies than a USB drive (which has to be connected and powered on all the time to do the backup file sync to the USB drive after the scheduled backup jobs). Reflect has its Image Guardian which acts like a rootkit by injecting its driver in the OS file I/O APIs to protect against writes and deletes of the backup files (but doesn't prevent reading them). Image Guardian helps a bit against ransomware for either the backup files on my D: "backup" internal HDD drive and the backup of backup files on the E: USB HDD drive. However, if I can figure out how to workaround Image Guardian, I'm sure malware can, too. I discussed the window of opportunity for malware to get past Image Guardian with Macrium, they agreed, but they weren't interested in a more thorough protection scheme. They added something to bolster their backup software against most ransomware, and it suffices for that. Anything that is currently mounted can be found by ransomware or malware. Hell, all it has to do is run diskpart and run "list volume". Drive letters are not required to get at partitions. Acronis, Paragon, and another (I forget which) can store backups in "hidden" partitions: the partition type number is non-standard, so standard disk tools should leave it alone, and no drive letter is assigned (which is the "hiding" part), but those backup programs can still access the partitions to write and read their backup files from there. Not assigning a drive letter to a partition doesn't block access to the partition, so it's a weak anti-ransomware protection scheme, but better than nothing. Programs can use mount points. Humans have been trained to recognize drive letters. With backup files getting copied to an FTP server, malware can't reach those backups because they won't know the login credentials for the account under which the backups get uploaded to the FTP host, and other modes of access would be locked out. For scheduled backup to run and then send a copy to the FTP server, the login credentials would have to be stored but be secure to prevent malware from finding the login credentials. FTPS would be needed to secure the login credentials during transit to prevent sniffing by malware. |
|
Thread Tools | |
Display Modes | Rate This Thread |
|
|