Duplicati

On 6/26/19 6:39 AM, VanguardLH wrote:
T wrote:

VanguardLH wrote:

T wrote:

Any of you guys use Duplicati? Your evaluation?

https://www.duplicati.com/

I am trying to figure out if its eMail utility will
handle OAuth2 (gmail, yahoo) before even trying it out.

https://duplicati.readthedocs.io/en/...anced-options/
Not seeing a way to use OAuth2 with eMailed reports, but ...

Whose e-mail service are you using that forces you to use OAUTH[2]?
Google is probably the largest purveyor of OAUTH2, but you can disable
that requirement in the server-side settings for your Gmail account.


"OAuth2 (gmail, yahoo)".

Yes, you can disable it on gMail and Yahoo, but the ass holes
keep sending out robo eMail telling the user to "Turn off
untrusted apps" and it "scares" the users, so they eventually
turn it off, despite what I say, and it breaks those clients
not using AOuth2.

This is a user problem as much as it is a technical problem.

Do you have an open source backup software that you like
that can handle OAuth2 eMail reports?

On 6/26/19 6:26 AM, VanguardLH wrote:
T wrote:

Have you used Duplicati? If so, what is your technical opinion
of it?

"Free backup software to store encrypted backups online"

It also works locally.

No thanks. Have you ever tried to restore a host via Internet? Slow!
Just trying to restore files is slow. Disasters often result due to
compounding effects. How are you going to restore when you also have an
Internet outage? Your network might be working just fine but not your
ISP, or the nodes in your route, or the endpoint (online storage).
Duplicati does support FTP, NAS, and other file transfers that can be on
your own network.

You are preaching to the choir. Cloud (client server with a really
s***ty connection between the two) is a disaster.


https://www.cloudwards.net/review/duplicati/

Duplicati is definitely something every tinkerer should look into.
People that want ready-to-go solutions might want to give it a pass.

Duplicati isn’t a standard backup service. It doesn’t come with backup
space, but instead provides you with a web client that lets you manage
your backup on a cloud service. That’s a blessing or a curse,
depending on your IT knowledge and willingness to tinker.

Because it’s free, it lacks a dedicated team of support technicians
ready to help solve your problems. That said, there’s an active forum
where you can ask for help.


Not an issue. I was not thinking of using it with a cloud service.
Maybe in some instances, to a local FTP server on a local area network,
which can really mess with ransomware's mind, write only directories
and such (they have to first read to encrypt).

Never heard of them until your post although Duplicati 2 (a complete
rewrite) is 7 years old yet it is still listed as betaware. If you were
thinking you could resort to getting support from Duplicati when you
customers have problems, you won't except for peer support and that
often is sporadic in response and irregular in quality. You claim to
have customers and you're their tech support, so why are you looking at
non-commercial products with no support? When you can't figure out a
resolution, your customers don't care and still want a problem fixed and
now. They can't be on hold waiting for restores, and that includes a
slow restore via Internet. For one file, maybe. For a few files,
maybe. For a host restore, NO.

While Duplicati, a client, may be free, looks like you have to pay for a
business service tier at the cloud file providers. After all, for
example, if you're using a free Dropbox account then all you have is 2GB
of online storage, and these days that is a pitiful quota for backups.
OneDrive and Google Drive give you 15GB, but that's still too small for
backing up a host. If you're operating your own FTP server then the
cost is your effort and for the software, but lots of backup programs
support FTP.

https://www.cloudwards.net/award/bes...ckup-services/
(click on "All Plans" for Duplicati)

Also, from that article, Duplicati does not support image backups, just
file backups. That means you have to [re]install the OS before you can
restore files to it. Image backups are directly restorable.

I really don't care about that. I just need file backup. I have
found with Windows that it is better to just reinstall everything
from scratch when a disaster strike that takes out their hard drive.
A full reinstall removes all the sins of the past. They tell
me it is like a brand new machine.

I have found it impossible to do a full restore of Windows
anyway -- too many file locks and such that get missed by even
the best backup software. A full restore requires a clone.
And a full reinstall removes so much trash (junkware, expired
software, DLL's, etc.) that a full backup really is not justified, as
it never reinstalls anyway.

Hell I have enough trouble backing up QuickBooks, even with volume
shadow copy. What? Turn QuickBooks off before leaving for the
night you say? Not going to happen. Requires too many fingers.
(Solution, do a reboot /f first, then do the backup, but I worry about
corrupting QuickBooks in the process.)

Customers will not do a clone anyway. It takes too long and they
would have to lift more that one finger.

Linux, I can restore every single piece of everything. It is a much
different backup and restore environment, but I am looking for an
open source Windows solution here. (I personally do
a clone of my Linux system once a week with Clonezilla.)

Duplicati does not seems all that maintained if they still
are only supporting OAuth1

On 6/25/19 4:14 PM, T wrote:
Hi All,

Any of you guys use Duplicati?Â* Your evaluation?

https://www.duplicati.com/

I am trying to figure out if its eMail utility will
handle OAuth2 (gmail, yahoo) before even trying it out.

Many thanks,
-T

Just got this back from their forum:

Backups use ZIP and AES256 standards but they are not
readable by other products. You will need to use Duplicati
to restore the data.

And that is a deal killer.

T wrote:

Â*Â*Â*Â*Â* Backups use ZIP and AES256 standards but they are not
Â*Â*Â*Â*Â* readable by other products. You will need to use Duplicati
Â*Â*Â*Â*Â* to restore the data.

And that is a deal killer.

Can Macrium Reflect restore a backup taken by Acronis True Image?
Can Data Protection Manager restore a backup taken by BackupExec?

Andy Burns wrote:
T wrote:

Backups use ZIP and AES256 standards but they are not
readable by other products. You will need to use Duplicati
to restore the data.

And that is a deal killer.

Can Macrium Reflect restore a backup taken by Acronis True Image?
Can Data Protection Manager restore a backup taken by BackupExec?

They discussed the idea of a mounter about a year ago.

https://github.com/duplicati/duplicati/issues/3081

Linux has FUSE and Windows has IFS, but I doubt the
Macrium mounter is IFS. They probably use some other
trick, to fool Disk Management (on the fly conversion
to some standard Disk Management understands).

Macrium, Acronis, and Ghost, all have mounter/viewer type
applications. Macrium would convert a MRIMG to a VHD back
in version 6, but version 7 dropped that feature. Viewer
applications allow random access to the restore data,
so you don't have to restore everything just to get
one file.

Paul

T wrote:

On 6/26/19 6:39 AM, VanguardLH wrote:
T wrote:

VanguardLH wrote:

T wrote:

Any of you guys use Duplicati? Your evaluation?

https://www.duplicati.com/

I am trying to figure out if its eMail utility will
handle OAuth2 (gmail, yahoo) before even trying it out.

https://duplicati.readthedocs.io/en/...anced-options/
Not seeing a way to use OAuth2 with eMailed reports, but ...

Whose e-mail service are you using that forces you to use OAUTH[2]?
Google is probably the largest purveyor of OAUTH2, but you can disable
that requirement in the server-side settings for your Gmail account.


"OAuth2 (gmail, yahoo)".

Yes, you can disable it on gMail and Yahoo, but the ass holes
keep sending out robo eMail telling the user to "Turn off
untrusted apps" and it "scares" the users, so they eventually
turn it off, despite what I say, and it breaks those clients
not using AOuth2.

I only got that e-mail once or twice, not repetitively. Besides, you
could define a server-side filter to delete that message. I just
checked my service-side filters in my Gmail account and don't see one
that targets the bogus "Your client is not secure" e-mail or anything
like it, so I haven't gotten more than a couple; else, I would've been
spurred to create such a filter. Um, besides, you are asking about
Duplicati *sending* e-mails, like status of backups. Does Duplicati
actually function like a full e-mail client and accept e-mails to
itself? I suspect the e-mail feature in Duplicati is outbound-only to
*send* reports. It sends e-mail, not receives e-mail (why would it need
to report to itself, especially since it has [should have] logs).

The users getting the e-mailed alerts can use whatever client they want,
including those that Google thinks are "secure" meaning they support
OAUTH2. However, as I said, I've gotten those "insecure" bogus alerts
only a couple times. It's been so few over the several years that
Google implement OAUTH2 and added the "allow insecure" setting that I
really can't say how many total that I've received over all that time.
No one can be that sensitive to a couple unwanted e-mails in a year.

You'll be waiting a long time to get the next Google e-mail about
insecure clients to get a sample upon which you can determine how to
define a filter, unless there's still one sitting in the Deleted folder.

This is a user problem as much as it is a technical problem.

Do you have an open source backup software that you like
that can handle OAuth2 eMail reports?

I think you had this discussion before. Aren't the one still using the
8-year unsupported Cobian backup software? If so, another requirement
you had back then for an alternative was the backups had to be in some
non-proprietary format (well, a common-enough file format that you
weren't forced to use the specific backup program to look inside the
backups), like saving into .zip files (although .rar or .7z would be
better - but creating .rar files costs money, so you're not going to
find freeware offering RAR compression, yet 7-zip provides the libs for
free and it and others using the 7z libs, like Peazip, are free and
support .7z compression). Neither .rar or .7z have taken off like .zip,
but there are lots of free tools to look into or extract from .7z files.

I've looked at free backup software for many years, like each time I get
motivated to get something better based on my criteria at the time. I
wasn't looking for open source software, just freeware. Even when I
decide on a freeware solution, I often end up wanting a feature
available only by upgrading to the payware version, like going from
Macrium Reflect Free to Macrium Reflect Home.

Any free (open source or not) backup software that lets you run post-job
commands would let you run 7-zip or Peazip using their CLI (command-line
interface). You could use any copy or sync program to copy the source
files to a destination folder and then compress that folder into an
archive. If you use a .bat file, it could do the compressed archive
from the destination folder specifed in the copy/sync program and then
rename the file with the current date (and maybe time). The batch file
could run the compression of the output folder and then use "date /t
for /f ..." and "time /t for /f ..." commands to get the date and time
to parse by the for command to use in renaming the archive file.

Of course, some file sync programs will .zip up the output. SyncBack
Free (nope, not open source) will copy the files to a destination folder
and then compress it into a .zip archive. Syncback lets you specify a
post-job command, but I don't remember if that's in its free version.
The free version also support FTP (but not FTPS).

Of course, all this tinkering is manual, requires testing and
validation, and the same reason why Duplicati gets panned by users
looking for an integrated product. SyncBackFree doesn't support VSC
(Volume Shadow Copy), so it cannot access locked/inuse files. No idea
what would be in the fileset you are backing up for your customers.

T wrote:

On 6/25/19 4:14 PM, T wrote:
Hi All,

Any of you guys use Duplicati?* Your evaluation?

https://www.duplicati.com/

I am trying to figure out if its eMail utility will
handle OAuth2 (gmail, yahoo) before even trying it out.

Many thanks,
-T

Just got this back from their forum:

Backups use ZIP and AES256 standards but they are not
readable by other products. You will need to use Duplicati
to restore the data.

And that is a deal killer.

My guess is you could open or extract from the .zip file but you'd get
garbage because of the encryption (and it sounds like they're not
passwording the .zip file -- which is weak protection, anyway -- so
whomever wants to extract out the .zip would have to know the encryption
key to decrypt the extracted files. They don't look like they are
designing a home-use product but something for enterprise deployment,
and businesses want their data protected, even when extracting out of a
compressed archive file.

VanguardLH wrote:
T wrote:

On 6/25/19 4:14 PM, T wrote:
Hi All,

Any of you guys use Duplicati? Your evaluation?

https://www.duplicati.com/

I am trying to figure out if its eMail utility will
handle OAuth2 (gmail, yahoo) before even trying it out.

Many thanks,
-T
Just got this back from their forum:

Backups use ZIP and AES256 standards but they are not
readable by other products. You will need to use Duplicati
to restore the data.

And that is a deal killer.

My guess is you could open or extract from the .zip file but you'd get
garbage because of the encryption (and it sounds like they're not
passwording the .zip file -- which is weak protection, anyway -- so
whomever wants to extract out the .zip would have to know the encryption
key to decrypt the extracted files. They don't look like they are
designing a home-use product but something for enterprise deployment,
and businesses want their data protected, even when extracting out of a
compressed archive file.

It's "zip then encrypt".

The entropy of encryption makes compression all but impossible.
That's why the Bitlocker encryption uses "elephant diffuser",
to make the data even more random looking.

As a result, you zip first, to squeeze out the fat, and then
the randomization of the encryption doesn't hurt anything
(1:1 byte ratio).

So rather than looking at a file ending in .zip, it's a file
ending in .sha256, with a ZIP completely hidden inside. Now,
they can always have an unencrypted header and an encrypted
payload, but you get the idea. There won't be much to see
from the outside, until a streaming decrypter can be set
running.

I got fooled on this, with some pretty good privacy tool
I was testing. I made some test files. I made an all-zeros
test file (to make it easier to detect what it was doing,
or so I thought). The tool I was testing, was careful not
to state what it was doing. I was sitting there waiting
for the disk drive light to go crazy, and there was
nothing. Turns out, it compressed the file full of zeros
down to practically nothing, and it took like a microsecond
to do the encryption on the tiny file. And I didn't even
notice the blip on the hard drive, when it was written out.
This is why I remember "ZIP then encrypt", because that's
what it was doing before applying RSA2048. And then it dawned
on me "of course, the stupid thing just compressed my file".

Paul

On Tue, 25 Jun 2019 16:14:55 -0700, T wrote:

Hi All,

Any of you guys use Duplicati? Your evaluation?

I`m using Duplicati (1.3.4) since 5 years or so, together with AWS. No
problems at all, stable from reboot to reboot (every few months).
Duplicati is running in an old XP-VM, which collects backups from my
network. And because it`s so stable, I havn`t upgraded XP or
Duplicati.
Testing restore on another machine (Win 10) was easy.

Regards

M.

On Wed, 26 Jun 2019 12:21:52 -0700, T wrote:

Just got this back from their forum:

Backups use ZIP and AES256 standards but they are not
readable by other products. You will need to use Duplicati
to restore the data.

And that is a deal killer.

Why? Duplicati works and it`s free. I assume, differential backups are
done a bit different from product to product, so it`s easy to
understand why you need Duplicati for a restore.

Michael Logies wrote:

I`m using Duplicati (1.3.4) since 5 years or so, together with AWS.

Must admit, I only tried tried v2.x

On 6/26/19 9:47 PM, VanguardLH wrote:
Aren't the one still using the
8-year unsupported Cobian backup software?

Yup. And I am looking for an alternative again. Must be

1) open source

2) have plain backups able to be read by any reader

3) eMail reports OAuth2 would be a plus

4) pre and post events.

On 6/27/19 11:54 AM, T wrote:
On 6/26/19 9:47 PM, VanguardLH wrote:
Aren't the one still using the
8-year unsupported Cobian backup software?

Yup.Â* And I am looking for an alternative again.Â* Must be

1) open source

2) have plain backups able to be read by any reader

3) eMail reports OAuth2 would be a plus

4) pre and post events.



I keep thinking I might have to break down and write this myself.
I really, really do not want to start that kind of project!
I just do not have the time

T wrote:
On 6/27/19 11:54 AM, T wrote:
On 6/26/19 9:47 PM, VanguardLH wrote:
Aren't the one still using the
8-year unsupported Cobian backup software?

Yup. And I am looking for an alternative again. Must be

1) open source

2) have plain backups able to be read by any reader

3) eMail reports OAuth2 would be a plus

4) pre and post events.



I keep thinking I might have to break down and write this myself.
I really, really do not want to start that kind of project!
I just do not have the time


"A bottomless hole opens and swallows Todd"

It's a good thing you stepped around that hole like that.

Paul

T wrote:
On 6/26/19 9:47 PM, VanguardLH wrote:
Aren't the one still using the
8-year unsupported Cobian backup software?

Yup. And I am looking for an alternative again. Must be

1) open source

2) have plain backups able to be read by any reader

3) eMail reports OAuth2 would be a plus

4) pre and post events.

If I were you, I would just continue to use Cobian Backup till it
breaks. (I sure will do so for our systems.)

As to your requirements:

1) open source

Should be irrelevant. Most other software your customer and you use
isn't open source either. (N.B. I've been doing Unix/UNIX/unix since
nearly four decades, but I'm no open source zealot (nor a free software
one).)

3) eMail reports OAuth2 would be a plus

Gmail alternative solution described in my other response.

2) and 4) are of course covered by Cobian Backup.

BTW, is the backup device always on/connected? If so, then why don't
you just use File History and be done with it? It wouldn't have 3) and
4), but there would be no need for those.