If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Rate Thread | Display Modes |
#31
|
|||
|
|||
RAR Files
On Mon, 24 Jun 2019 17:47:24 -0400, Paul wrote:
To add reliability to any packaging, you can use Parchive techniques from the outside. I don't know if anyone has put this into, say, a multi-part archive as a means to improving reliability. https://en.wikipedia.org/wiki/Parchive Well, like you said, par files are computed after the fact, when the archive has already been created. That was also proposed as a mechanism for enhancing data stored on CDs or DVDs, but the problem with that concept, is when the "error rate" gets high enough on optical media, the laser refuses to track the spiral, and you can't pull anything off the disc. So Parchive for optical media is a waste of time, just because of error characteristic (tolerates errors well, until the laser loses lock). I wouldn't call it a waste of time. You don't have to wait until the read error rate gets "high enough", but if you do want to plan for that scenario then just store your data in two places rather than one. You should probably do that anyway. Two copies of data and one set of par files will protect you very well, if not a bit of overkill. In most cases, if you're going to use obsolete storage methods such as optical discs, you could consider putting the data files on one disc and the par files on another disc. Test it once in a while, (using one of the par files, of course), to check for bit rot. And that leaves PAR as a mechanism for storing, say, chunks on multiple drives. RAID without a RAID. You could have four separate hard drives, store a three chunk file, generate a fourth PAR file, and now, if one of the four drives goes down, the missing PAR chunk is replaced by one other chunk. That's the right idea, but that would be a very strange implementation. Data files don't know about chunks or par files, and there certainly wouldn't be any reason to store each data file on a different drive. When you create the par files, you get to decide how many chunks you want the par files to have, but the data files don't know anything about that. When it comes to storage, generally the par files accompany the data files but that's not a hard requirement. I played with PAR a bit at the time, but one of the problems I was finding in forums, is it wasn't mathematically robust. Some people were unable to recover damaged archives using the additional PAR blocks they had on hand, and it was claimed to be a "matrix problem", because the technique involves large sparse matrices. That sounds suspiciously like a par1 issue. By 2002 and the advent of par2, issues like that disappeared and it's been robust ever since. I've been using par files heavily since they were introduced, (we routinely transfer files for remote work purposes and needed a way to verify integrity and optionally repair damage), and I can't remember the last time I saw a par issue. If there was one, it probably would have been in 2001, before par2 hit the scene. |
Ads |
#32
|
|||
|
|||
RAR Files
On 24/06/2019 23.47, Paul wrote:
Carlos E.R. wrote: On 24/06/2019 20.10, Paul wrote: Carlos E.R. wrote: On 24/06/2019 04.59, VanguardLH wrote: Paul wrote: [RAR] uses the old ZIP model, where decompression was free, and compression costs money. That's why plugins are available for decompression of RAR, which other designers can incorporate. RARLAB's UnRAR is open source.Â* Anyone can use the free code libraries to decompress RAR. I always thought it presumptuous or even stupid that RARLAB charges for their proprietary RAR program.Â* Although claimed better, does RAR have a command lead in usage and is it really that much better than .zip or .7z that it can command a commercial price ($29 USD)? RAR has some advantages. It includes verification, but optionally can include repair info. That is, a damaged archive can be repaired. This makes it good for backups. It can easily create split archives, filling some destination media (initially, floppies). It can run a script (batch) when it is decompressed, which is useful for creating installable programs. I did not know the price is so much. $29? On the other hand, it does not support the full attribute set of a Linux filesystem. Ie, is not really multiplatform. Many compression schemes support verification. But last time I looked none had "repair" in a stable release considered reliable. To add reliability to any packaging, you can use Parchive techniques from the outside. I don't know if anyone has put this into, say, a multi-part archive as a means to improving reliability. I know about par, I have used par2 on DVDs. But not on archives. https://en.wikipedia.org/wiki/Parchive That was also proposed as a mechanism for enhancing data stored on CDs or DVDs, but the problem with that concept, is when the "error rate" gets high enough on optical media, the laser refuses to track the spiral, and you can't pull anything off the disc. So Parchive for optical media is a waste of time, just because of error characteristic (tolerates errors well, until the laser loses lock). Right :-( Question. Does proper backup to DVD software exists on Windows, that do error prevention and recovery? And that leaves PAR as a mechanism for storing, say, chunks on multiple drives. RAID without a RAID. You could have four separate hard drives, store a three chunk file, generate a fourth PAR file, and now, if one of the four drives goes down, the missing PAR chunk is replaced by one other chunk. I played with PAR a bit at the time, but one of the problems I was finding in forums, is it wasn't mathematically robust. Some people were unable to recover damaged archives using the additional PAR blocks they had on hand, and it was claimed to be a "matrix problem", because the technique involves large sparse matrices. This is a kind of error correcting code, which is likely still in usage every day, for movie downloads. You "spray" a server with twice as many segments as is needed to represent a movie, the movie company does DMCA takedowns on the articles, and if enough survive, PAR allows them to be glued together to make an intact movie. It's something along those lines. Maybe you have 1,2,3,29,30,31,1000,1008,1023... when the movie is 900 long, and the movie can still be recovered as you download more than 900 remaining segments. Curious. Still, par is extra work, for backups. I like the idea of rar doing the creation of the recovery data internal, and then the repair when needed. I mean, this is a serious advantage for rar. It is not the case in Windows, but you know that a single byte changed in a tgz destroys it completely. Possibly other formats, used in Windows, suffer the same risk. Not rar. -- Cheers, Carlos. |
#33
|
|||
|
|||
RAR Files
On 25/06/2019 17.26, Char Jackson wrote:
On Mon, 24 Jun 2019 17:47:24 -0400, Paul wrote: To add reliability to any packaging, you can use Parchive techniques from the outside. I don't know if anyone has put this into, say, a multi-part archive as a means to improving reliability. https://en.wikipedia.org/wiki/Parchive Well, like you said, par files are computed after the fact, when the archive has already been created. That was also proposed as a mechanism for enhancing data stored on CDs or DVDs, but the problem with that concept, is when the "error rate" gets high enough on optical media, the laser refuses to track the spiral, and you can't pull anything off the disc. So Parchive for optical media is a waste of time, just because of error characteristic (tolerates errors well, until the laser loses lock). I wouldn't call it a waste of time. You don't have to wait until the read error rate gets "high enough", but if you do want to plan for that scenario then just store your data in two places rather than one. You should probably do that anyway. Two copies of data and one set of par files will protect you very well, if not a bit of overkill. In most cases, if you're going to use obsolete storage methods such as optical discs, you could consider putting the data files on one disc and the par files on another disc. Test it once in a while, (using one of the par files, of course), to check for bit rot. Backup to tape is too expensive for home usage. And there is BlueRay. -- Cheers, Carlos. |
#34
|
|||
|
|||
RAR Files
On Wed, 26 Jun 2019 12:32:29 +0200, "Carlos E.R."
wrote: Question. Does proper backup to DVD software exists on Windows, that do error prevention and recovery? That's a conundrum. Can backup software be considered "proper" if it's backing up to DVD? Didn't backup to optical media become obsolete something like 15+ years ago for DVD, and even longer ago for CD-R? |
#35
|
|||
|
|||
RAR Files
On Wed, 26 Jun 2019 12:35:23 +0200, "Carlos E.R."
wrote: On 25/06/2019 17.26, Char Jackson wrote: I wouldn't call it a waste of time. You don't have to wait until the read error rate gets "high enough", but if you do want to plan for that scenario then just store your data in two places rather than one. You should probably do that anyway. Two copies of data and one set of par files will protect you very well, if not a bit of overkill. In most cases, if you're going to use obsolete storage methods such as optical discs, you could consider putting the data files on one disc and the par files on another disc. Test it once in a while, (using one of the par files, of course), to check for bit rot. Backup to tape is too expensive for home usage. And there is BlueRay. Personally, I haven't seen a tape backup system in use since 1996, whether it's in a home environment or an Enterprise environment. As for BluRay, it's part of the optical family and appeared on the scene when optical was already in sharp decline. BluRay wasn't able to reverse that trend. |
#36
|
|||
|
|||
RAR Files
Char Jackson wrote:
Carlos E.R. wrote: Backup to tape is too expensive for home usage Personally, I haven't seen a tape backup system in use since 1996 I know of a system that backed-up daily to DC6150 tapes until about 4 years ago. |
#37
|
|||
|
|||
RAR Files
On Wed, 26 Jun 2019 18:41:07 +0100, Andy Burns
wrote: Char Jackson wrote: Carlos E.R. wrote: Backup to tape is too expensive for home usage Personally, I haven't seen a tape backup system in use since 1996 I know of a system that backed-up daily to DC6150 tapes until about 4 years ago. A government system? They seem to usually be the last holdouts on obsolete technologies. With their limited budgets, it's not sexy to spend money on backup systems so other things get higher priority. |
#38
|
|||
|
|||
RAR Files
Char Jackson wrote:
Andy Burns wrote: I know of a system that backed-up daily to DC6150 tapes until about 4 years ago. A government system? Yep, fire service control room system |
#39
|
|||
|
|||
RAR Files
On 26/06/2019 19.32, Char Jackson wrote:
On Wed, 26 Jun 2019 12:35:23 +0200, "Carlos E.R." wrote: On 25/06/2019 17.26, Char Jackson wrote: I wouldn't call it a waste of time. You don't have to wait until the read error rate gets "high enough", but if you do want to plan for that scenario then just store your data in two places rather than one. You should probably do that anyway. Two copies of data and one set of par files will protect you very well, if not a bit of overkill. In most cases, if you're going to use obsolete storage methods such as optical discs, you could consider putting the data files on one disc and the par files on another disc. Test it once in a while, (using one of the par files, of course), to check for bit rot. Backup to tape is too expensive for home usage. And there is BlueRay. Personally, I haven't seen a tape backup system in use since 1996, whether it's in a home environment or an Enterprise environment. As for BluRay, it's part of the optical family and appeared on the scene when optical was already in sharp decline. BluRay wasn't able to reverse that trend. I know enterprise people doing backup to tape often, today. They use LTO. The units are very expensive, starting at 1500€ for an LTO-6 -- Cheers, Carlos. |
#40
|
|||
|
|||
RAR Files
On 26/06/2019 19.25, Char Jackson wrote:
On Wed, 26 Jun 2019 12:32:29 +0200, "Carlos E.R." wrote: Question. Does proper backup to DVD software exists on Windows, that do error prevention and recovery? That's a conundrum. Can backup software be considered "proper" if it's backing up to DVD? Didn't backup to optical media become obsolete something like 15+ years ago for DVD, and even longer ago for CD-R? Blueray. Besides external hard disks stored in a safe box, what else is available for home users? -- Cheers, Carlos. |
#41
|
|||
|
|||
RAR Files
Carlos E.R. wrote:
On 26/06/2019 19.32, Char Jackson wrote: On Wed, 26 Jun 2019 12:35:23 +0200, "Carlos E.R." wrote: On 25/06/2019 17.26, Char Jackson wrote: I wouldn't call it a waste of time. You don't have to wait until the read error rate gets "high enough", but if you do want to plan for that scenario then just store your data in two places rather than one. You should probably do that anyway. Two copies of data and one set of par files will protect you very well, if not a bit of overkill. In most cases, if you're going to use obsolete storage methods such as optical discs, you could consider putting the data files on one disc and the par files on another disc. Test it once in a while, (using one of the par files, of course), to check for bit rot. Backup to tape is too expensive for home usage. And there is BlueRay. Personally, I haven't seen a tape backup system in use since 1996, whether it's in a home environment or an Enterprise environment. As for BluRay, it's part of the optical family and appeared on the scene when optical was already in sharp decline. BluRay wasn't able to reverse that trend. I know enterprise people doing backup to tape often, today. They use LTO. The units are very expensive, starting at 1500€ for an LTO-6 It's pretty simple. Look at the price of certified tape, versus the price of a hard drive you can trust. And in terms of characteristics, IT people know how tape works, what maintenance it requires (rewind once in a while, for the cartridges, run a cleaning cassette once in a while on the drive). The tape wouldn't be destroyed by an EMP. The fastest tape drive I ever heard of, was 1GB/sec, but the cartridge for that, was a joke. Saw the sample at a conference. I thought the cartridge was a stage prop at first, because it was so big. On tape farms, a layer of HDD caches are used. You run your backups in the wee hours, with the expected "window" for your incrementals, and the images are stored on the hard drive, until the tape autoloader can copy the materials. With a good autoloader, all what you need is someone labeling the materials before storage. If the tapes had an RFID tag, or a bar code, even that could be automated. Paul |
#42
|
|||
|
|||
RAR Files
Carlos E.R. wrote:
On 26/06/2019 19.25, Char Jackson wrote: On Wed, 26 Jun 2019 12:32:29 +0200, "Carlos E.R." wrote: Question. Does proper backup to DVD software exists on Windows, that do error prevention and recovery? That's a conundrum. Can backup software be considered "proper" if it's backing up to DVD? Didn't backup to optical media become obsolete something like 15+ years ago for DVD, and even longer ago for CD-R? Blueray. Besides external hard disks stored in a safe box, what else is available for home users? The largest BluRay (write-once) is 100GB. The media is expensive (and the pricing is based on how many DVDs it replaces). And you can get re-writable media. https://www.verbatim.com/subcat/opti...blu-ray/bd-re/ And I learned something new today... Verbatim is owned by Mitsubishi Chemical. ******* Example of archival BD materials. https://www.amazon.ca/Verbatim-M-Dis.../dp/B011PIJPOC Paul |
#43
|
|||
|
|||
RAR Files
In article , Carlos E.R.
wrote: Besides external hard disks stored in a safe box, what else is available for home users? a local nas, optionally (and ideally) synced to the cloud. |
#44
|
|||
|
|||
RAR Files
On Wed, 26 Jun 2019 12:32:41 -0500, Char Jackson
wrote: On Wed, 26 Jun 2019 12:35:23 +0200, "Carlos E.R." wrote: On 25/06/2019 17.26, Char Jackson wrote: I wouldn't call it a waste of time. You don't have to wait until the read error rate gets "high enough", but if you do want to plan for that scenario then just store your data in two places rather than one. You should probably do that anyway. Two copies of data and one set of par files will protect you very well, if not a bit of overkill. In most cases, if you're going to use obsolete storage methods such as optical discs, you could consider putting the data files on one disc and the par files on another disc. Test it once in a while, (using one of the par files, of course), to check for bit rot. Backup to tape is too expensive for home usage. And there is BlueRay. Personally, I haven't seen a tape backup system in use since 1996, whether it's in a home environment or an Enterprise environment. I can't remember what year it was the last time I used one, but it was around the same time. |
#45
|
|||
|
|||
RAR Files
On Wed, 26 Jun 2019 20:08:48 +0200, "Carlos E.R."
wrote: On 26/06/2019 19.25, Char Jackson wrote: On Wed, 26 Jun 2019 12:32:29 +0200, "Carlos E.R." wrote: Question. Does proper backup to DVD software exists on Windows, that do error prevention and recovery? That's a conundrum. Can backup software be considered "proper" if it's backing up to DVD? Didn't backup to optical media become obsolete something like 15+ years ago for DVD, and even longer ago for CD-R? Blueray. Besides external hard disks stored in a safe box, what else is available for home users? Web-based backup, such as Carbonite. |
Thread Tools | |
Display Modes | Rate This Thread |
|
|