Welcome!
Important information
-- Spectre and Meltdown vulnerabilities
-- Change in MX sources

News
-- MX Linux on social media: here
-- Mepis support still here

Current releases
-- MX-17.1 Final release info here
-- antiX-17 release info here

New users
-- Please read this first, and don't forget to add system and hardware information to posts!
-- Here are the Forum Rules

[SOLVED] 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

Message
Author
User avatar
MX-16_fan
Forum Regular
Forum Regular
Posts: 957
Joined: Mon Feb 13, 2017 12:09 pm

[SOLVED] 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#1 Post by MX-16_fan » Thu Jul 26, 2018 10:12 am

@all:

I am currently burning some files on DVD for a friend who wants to keep this DVD for "long-term" storage.

I am going to employ dvdisaster to add a layer of protection against data loss.

In order to be able to maintain original file names and original file creation dates, I want to put the files into an archive.

Considering that DVDs don't last forever, and that even minor scratches on DVDs can lead to massive data loss, I was asking myself which archive file format in this context might provide the highest level of error-tolerance.

I'd guess one can say that
would probably be the best group of options in this case.

But which of those four would be the least vulnerable – what do you think?


Greetings, and thanks in advance for your assessment, Joe
Last edited by MX-16_fan on Sun Jul 29, 2018 11:16 am, edited 1 time in total.

sunrat
Forum Regular
Forum Regular
Posts: 219
Joined: Mon Mar 28, 2016 9:54 pm

Re: 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#2 Post by sunrat » Thu Jul 26, 2018 10:47 am

MX-16_fan wrote:
Thu Jul 26, 2018 10:12 am
But which of those four would be the least vulnerable – what do you think?
I doubt there would be any significant difference between archive formats. The reliability of the DVD media is the most important factor.
- Use a good archive quality DVD, not a cheapy from the corner shop.
- Make at least 2 copies.

User avatar
Adrian
Forum Veteran
Forum Veteran
Posts: 8918
Joined: Wed Jul 12, 2006 1:42 am

Re: 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#3 Post by Adrian » Thu Jul 26, 2018 10:55 am

I wonder if non compressed files are better (like tar) if the file is compressed a number of errors might render the file useless, while in a simple format you'd miss only the info that was destroyed, it won't destroy the entire file. I'm just guessing, I don't have a lot experience with recovering from compressed files.

User avatar
timkb4cq
Forum Veteran
Forum Veteran
Posts: 4370
Joined: Wed Jul 12, 2006 4:05 pm

Re: 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#4 Post by timkb4cq » Thu Jul 26, 2018 10:57 am

For the worst case scenario, tar + par2. The reason is that if a compressed archive is corrupt it can't always be decompressed. Software like photorec can retrieve files from a damaged tar if it's uncompressed.

Dar is interesting because it has a command line switch to automatically generate par2 files for archive recovery in case of data loss on the optical media. Combined with dvdisaster (which creates similar data on the image rather than on the files) you've got a pretty robust strategy. Especially if you burn multiple copies. In case of massive data loss, utilities like ddrescue can combine the readable data from multiple copies to assemble a good copy.
MSI 970A-G43 MB, AMD FX-6300 (six core), 16GB RAM, GeForce 730, Samsung 850 EVO 250GB SSD, Seagate Barracuda XT 3TB

User avatar
thomasl
Forum Novice
Forum  Novice
Posts: 92
Joined: Sun Feb 04, 2018 10:26 am

Re: 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#5 Post by thomasl » Thu Jul 26, 2018 11:07 am

I know it's not on your list but I swear by RAR and its redundancy/error correction features. I have been using this for decades, among other things exactly for the scenario you describe, and I've never had any trouble with any RAR archive.

There's a fully-featured Linux command line version available but it's not free.
Dual-boot MX17.1/64 frugal root persistence + Windows 7 on Lenovo Edge72 i5-3470S/12GB and Tosh R950 i5-3340M/8GB
“In foreign countries they fear baldness. They are so rich in foreign countries, they can afford to fear all kinds of silly things.”

User avatar
ChrisUK
Forum Regular
Forum Regular
Posts: 190
Joined: Tue Dec 12, 2017 1:04 pm

Re: 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#6 Post by ChrisUK » Thu Jul 26, 2018 11:14 am

Based on my personal experience, not on any tech info:

I've branded (TDK, Sony, Memorex, etc.) DVDs that haven't lasted 10 years... burned and stored, then checked less than 10 years later. I've old Hard Drives (PATA IDE) that have lasted over 20 years, and are still used often - if I wanted to store files "long term", I'd probably avoid DVDs. But I'd guess that others could post the opposite case ;)
Chris

MX 17 - Manjaro

User avatar
MX-16_fan
Forum Regular
Forum Regular
Posts: 957
Joined: Mon Feb 13, 2017 12:09 pm

Re: 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#7 Post by MX-16_fan » Thu Jul 26, 2018 11:59 am

@Adrian:
Adrian wrote:
Thu Jul 26, 2018 10:55 am
I wonder if non compressed files are better (like tar) if the file is compressed a number of errors might render the file useless ... .
That's what I was thinking about, too. However, tar alone at first glance seems to be rather unsafe, too, as by default it puts everything into one big file. I might be wrong, however. Maybe tar's checking mechanisms are better than I think.


Greetings, Joe

User avatar
MX-16_fan
Forum Regular
Forum Regular
Posts: 957
Joined: Mon Feb 13, 2017 12:09 pm

Re: 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#8 Post by MX-16_fan » Thu Jul 26, 2018 12:01 pm

@timkb4cq:
timkb4cq wrote:
Thu Jul 26, 2018 10:57 am
(...)
Dar is interesting because it has a command line switch to automatically generate par2 files for archive recovery in case of data loss on the optical media. Combined with dvdisaster (which creates similar data on the image rather than on the files) you've got a pretty robust strategy. (...)
Great hint, thanks!


Greetings, Joe

User avatar
MX-16_fan
Forum Regular
Forum Regular
Posts: 957
Joined: Mon Feb 13, 2017 12:09 pm

Re: 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#9 Post by MX-16_fan » Thu Jul 26, 2018 12:57 pm

@timkb4cq:
timkb4cq wrote:
Thu Jul 26, 2018 10:57 am
For the worst case scenario, tar + par2. (...)
Thanks! Just found a nice GUI for par2. It's in the repos. It's called "PyPar2".

Will try out par2 + tar.


Greetings, Joe

User avatar
MX-16_fan
Forum Regular
Forum Regular
Posts: 957
Joined: Mon Feb 13, 2017 12:09 pm

Re: 7z, DAR, tar, tar.gz: Which is safest for "long-term" storage of data on an optical disk?

#10 Post by MX-16_fan » Thu Jul 26, 2018 7:44 pm

@timkb4cq,
@all:

Thanks to everyone for his recommendations.

Here's the routine I finally worked out – might be interesting for some of you:

1) Produced a tar archive using file-roller.
2) Protected it using PyPar2.
3) Produced an ISO using Brasero.
4) Protected it using dvdisaster (with the recovery data on the DVD).
5) Burned the whole thing using Brasero.

This produced a DVD that MX-17 wouldn't even mount.

Seems to me there is a massive bug in dvdisaster: dvdisaster has a default concerning the DVD's size. There's an option for re-adjusting that to your actual empty DVD. I used it, and I think it's buggy in a way.

I noted that the 0.72.4-2 version of dvdisaster which Debian provides is way outdated. Maybe that error would vanish if the most current version were used. That would be 0.79.5 as of 2015. I'll file a separarate package request for that.

Generally, it might be a smart idea to integrate steps 1 to 4 into Brasero or some other CD/DVD burning application. One might call this the "Safety-enhanced archiving mode" or so.


Thanks to everyone, greetings, Joe

Post Reply

Return to “Software / Configuration”