Taperssection.com

Gear / Technical Help => Post-Processing, Computer / Streaming / Internet Devices & Related Activity => Topic started by: ThePiedPiper on September 12, 2019, 01:25:35 AM

Title: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: ThePiedPiper on September 12, 2019, 01:25:35 AM
What is your archival format preference?
Wav | Flac | Shn | Zip | Ape

Follow-up question:
In addition to the formatting listed above, which checksum is your preference (MD5 | FFP | SFV | CFP | ST5)?

Here's a rudimentary test I did with the following formats:

509mb APE (Level 5 "Insane")
526mb FLAC (Level 8)
537mb SHN
627mb ZIP (of Original Wav File)
678mb Wav

This may (more than likely) have been covered on here before, but I'm very curious to read the comments of "who, what, where, when & why".
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: ThePiedPiper on September 12, 2019, 01:59:13 AM
Here's some more on this subject. I found it very interesting.

https://wiki.hydrogenaud.io/index.php?title=Lossless_comparison
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: jerryfreak on September 12, 2019, 05:20:33 AM
FLAC

i havent used .ape since prob 2003
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: DavidPuddy on September 12, 2019, 10:22:07 AM
FLAC level 8
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: Gordon on September 12, 2019, 10:31:54 AM
flac 8 with ffp.  with ffp you can change tags etc and it will still check integrity.  with md5 if you change anything the md5 will fail.

Your flac size vs wav looks off to me.  I just checked one untracked master (24/48) and flac is 465mb, wav is 723mb
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: jefflester on September 12, 2019, 11:44:15 AM
Your flac size vs wav looks off to me.  I just checked one untracked master (24/48) and flac is 465mb, wav is 723mb
The amount of compression depends on the dynamics/level of the WAV that is being compressed. If the WAV is normalized, without many quiet sections it won't compress as much as a WAV that has a lot of dynamics and or doesn't fully peak. For example, I just took a pretty quiet 24/48 master that is 2:15 long.

master WAV: 1.99 GB
master WAV amplified by 10 dB: 1.99 GB (same, as it should be)
FLAC of master WAV: 1.15 GB (ratio 0.576)
FLAC of 10 dB amplified version: 1.28 GB (ratio 0.645)

The amplified version doesn't compress as much.

As another example, in Adobe Audition I just created 20 min of white noise and normalized it to 0 dB
WAV: 201 MB
FLAC: 196 MB (ratio 0.973)

Because the WAV file was basically full (constant near peak) it barely compressed at all.

I would assume the other lossless compression CODECs work similarly.

PiedPiper's example is likely constant and loud with little dynamics.

Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: Gordon on September 12, 2019, 12:16:17 PM
this is mostly loud rock music aud and sbd mixed to near 0db.  are you using the latest flac in your test?  I'm using the newest TLH that has the most current flac.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: heathen on September 12, 2019, 01:52:21 PM
For me shn and ape aren't contenders because they're proprietary/closed source.  I do still have a fair amount of older stuff in shn format, though, because that's how it was originally distributed.

I prefer flac and haven't come across a reason to explore other options.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: ThePiedPiper on September 12, 2019, 02:28:33 PM
flac 8 with ffp.  with ffp you can change tags etc and it will still check integrity.  with md5 if you change anything the md5 will fail.

Your flac size vs wav looks off to me.  I just checked one untracked master (24/48) and flac is 465mb, wav is 723mb

Nothing "off" about it. You are comparing the compression of a 24/48 with a 16/44.1, and as explained already, dynamics play a part of the equation.  I also said "rudimentary test" ... results always vary.

Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: Fatah Ruark (aka MIKE B) on September 12, 2019, 02:40:55 PM
I archive my master recordings exactly as they came out of the recorder (WAV).

I also keep the untracked final master in WAV.

Tracked out and tagged recordings are FLAC level 8. I have also converted any old SHN filesets to FLAC.

I'm not sure if my thoughts are accurate, but I'd rather not FLAC my master recordings since that adds one more thing that could go wrong (FLAC gets corrupted and won't unpack). I guess the WAV could have the same fate, but I do recall having an issue converting FLAC > WAV at some point.

Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: ThePiedPiper on September 12, 2019, 03:28:50 PM
I archive my master recordings exactly as they came out of the recorder (WAV).

I also keep the untracked final master in WAV.

Tracked out and tagged recordings are FLAC level 8. I have also converted any old SHN filesets to FLAC.

I'm not sure if my thoughts are accurate, but I'd rather not FLAC my master recordings since that adds one more thing that could go wrong (FLAC gets corrupted and won't unpack). I guess the WAV could have the same fate, but I do recall having an issue converting FLAC > WAV at some point.

I'm liking this idea! Thank you.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: beatkilla on September 12, 2019, 03:43:22 PM
I archive my master recordings exactly as they came out of the recorder (WAV).

I also keep the untracked final master in WAV.


Tracked out and tagged recordings are FLAC level 8. I have also converted any old SHN filesets to FLAC.

I'm not sure if my thoughts are accurate, but I'd rather not FLAC my master recordings since that adds one more thing that could go wrong (FLAC gets corrupted and won't unpack). I guess the WAV could have the same fate, but I do recall having an issue converting FLAC > WAV at some point.


Same here.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: Gutbucket on September 12, 2019, 04:03:38 PM
Your flac size vs wav looks off to me.  I just checked one untracked master (24/48) and flac is 465mb, wav is 723mb
The amount of compression depends on the dynamics/level of the WAV that is being compressed.

FWIW, Compression ratio also varies with regards to a number of other factors.  The particulars of lossless compression algorithms vary and can make for a rather complex mix of pattern-finding and data-reduction coding.

For instance, phase, level, and frequency correlation between channels is a big one.  A monophonic "stereo file" with identical information in both left and right channels will compress way more than a stereo file with a Taylor Swift in the left channel and a the Melvins in the right channel.  Less dramatically, I've notice that a coincident stereo recording compresses much more than a spaced omni recording of the same concert with similar levels.  The spaced omni recording has a lot of phase difference between channels, whereas the coincident stereo file has no phase difference (if using cardioids).. or at most some degree of polarity difference (if using mics with a inverse polarity rear lobe).
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: heathen on September 12, 2019, 04:56:36 PM
I archive my master recordings exactly as they came out of the recorder (WAV).

I do that too...forgot to mention that.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: IMPigpen on September 12, 2019, 04:58:50 PM
I archive my master recordings exactly as they came out of the recorder (WAV).

I do that too...forgot to mention that.

Ditto for me.  I keep the untouched WAV and also the final FLAC (Level 8) for archiving.  That way I can always go back to the raw WAVs and re-master/change it if needed in the future.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: freeform54 on February 04, 2021, 11:01:02 PM
I like FLAC Lossless Uncompressed - it's a setting on dBpoweramp a notch above FLAC level 8.  How is that different from WAV?
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: dabbler on February 03, 2022, 03:06:42 AM
FLAC all the way.  FLAC compression level doesn't matter as far as data integrity is concerned.  Higher compression levels use more CPU to encode/decode but also gives the smallest file sizes.  Low compression levels are the fastest to encode/decode.  On most computers, the CPU usage is negligible, so I opt for the smallest file size to (marginally) reduce bandwidth for backups and reduce SSD/HDD wear.

You can losslessly go from FLAC to WAV to FLAC again at any compression level, any number of times; as long as your memory/CPU/motherboard isn't corrupting data.  I always use workstations with ECC memory to
avoid data corruption.

WAV doesn't have built-in checksums, so you'll need to manage them yourself and/or rely on CRC from ZIP or some other method to notice bitrot as HDD/SSDs age and fail.  Having a filesystem which checksums data (e.g. ZFS, BTRFS) also helps.

If you get a corrupt FLAC file and no good backups; the "flac" command-line tool has a "--decode-through-errors" switch.  Having good backups is better, obviously.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: capnhook on February 03, 2022, 05:07:01 AM
I like FLAC Lossless Uncompressed - it's a setting on dBpoweramp a notch above FLAC level 8.  How is that different from WAV?

Welcome to taperssection, freeform54.  Lots of good stuff here.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: u2_fly_2 on February 05, 2022, 12:03:35 PM
I keep the original WAV-file and then make songsplit and put it out in FLAC Level 8.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: jb63 on February 05, 2022, 03:05:54 PM
I keep the original WAV-file and then make songsplit and put it out in FLAC Level 8.

I do this but instead flac level 5. I don't know why.
I can't seem to bring myself to delete that long tracked wav or tiff file, even if its 2496. Its just too pretty, even if it is in the way.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: morst on February 05, 2022, 03:28:27 PM
I keep the original WAV-file and then make songsplit and put it out in FLAC Level 8.

I do this but instead flac level 5. I don't know why.
I can't seem to bring myself to delete that long tracked wav or tiff file, even if its 2496. Its just too pretty, even if it is in the way.
I think 5 was the default.
FLAC encoding level 8 takes more processing power to make a slightly smaller file than lower numbers.
Back when computers were not as fast it was probably handy to use lower (faster) levels, but I suggest to stick with level 8 now.
Title: Re: What is your archival format preference? (Wav | Flac | Shn | Zip | Ape)
Post by: tim in jersey on March 06, 2022, 11:36:20 PM
Wax cylinders.