Taperssection.com
Gear / Technical Help => Ask The Tapers => Topic started by: mmmatt on October 20, 2005, 11:52:00 AM
-
I've been having major flac issues the past month or two. I flac all my master files to save disc space for storage. I typically tic the delete original files option because flac verifies before deleting. I have recently discovered that many of my masters are now shit. They won't uncompress and the flac verify feature didn't come up with any errors. I'm now not using the delete original feature but I still can't flac. I watch the dos windo and all verify OK, mthen I go back and test the files and half of them are bunk.
Anybody ever had this or have any ideas to fix?
Matt
-
???
better media ???
-
I flac all my master files to save disc space for storage. I typically tic the delete original files option because flac verifies before deleting. I have recently discovered that many of my masters are now shit.
Are the masters you're using on your harddrive or CD/DVD?
If they're on external media, it's likely a bad burn. If they're still on your HDD, I've got no clue
-
on the hard drive... I'm stumped. I've had a few problems burning straingt from flac in nero, and I've been having troubles with shn too, and my wavelab is acting wierd. Is there an f-u audio junky virus out there?
Matt
-
how old is the hard drive?
could be the HD dying if you are having problems with all those other applications.
just a guess though.
-
drive is over a year but no other problems. In this case I just flac'd a wav and it verified ok in the compression. then I take the flac to the program and test. Here is what I get:
flac 1.1.2, Copyright (C) 2000,2001,2002,2003,2004,2005 Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.
Bella Trio 2005-07-23 early show.flac: testing, 19% completeBella Trio 2005-07-2
3 early show.flac: *** Got error code 2:FLAC__STREAM_DECODER_ERROR_STATUS_FRAME_
CRC_MISMATCH
Bella Trio 2005-07-23 early show.flac: ERROR while decoding data
state = FLAC__STREAM_DECODER_READ_FRAME
-------------------------------------------------------------------------------
Press any key to continue . . .
This is fucked up
matt
-
I'd get your data backed up immediately and run HD diagnostics - I bet it's on the way down. Edit to add: run diagnostics on your RAM, too.
-
I've been archiving in .WAV format lately. A while ago there was a thread on the Oade site that got me thinking...
If you archive in FLAC and there is one small issue with the file, it will not decode. But, if you archive in .WAV you may be able to fix a problem if the file gets corrupted. FLAC is good, because it shrinks files for BT or download, so it easy to share over the internet. I don't think it's the best thing to use to archive however.
Let us know what you find. +T for your troubles.
-
Chuck, etc. while shorten will not continue to decode a file once an error is detected, Flac provides the option to decode through an error ( -F,--decode-through-errors)
Per FLAC documentation -- "By default flac stops decoding with an error and removes the partially decoded file if it encounters a bitstream error. With -F, errors are still printed but flac will continue decoding to completion. Note that errors may cause the decoded audio to be missing some samples or have silent sections."
FYI . .. .
Darrin
-
I have the decode through errors turned off. I'm going to try removing and reloading an older version of flac... seems this problem started about the time aI updated.
Good call on the backup brian... that will be step 2. Sure would be easier if I could flac these friggin files though!
Matt
-
Chuck, etc. while shorten will not continue to decode a file once an error is detected, Flac provides the option to decode through an error ( -F,--decode-through-errors)
Per FLAC documentation -- "By default flac stops decoding with an error and removes the partially decoded file if it encounters a bitstream error. With -F, errors are still printed but flac will continue decoding to completion. Note that errors may cause the decoded audio to be missing some samples or have silent sections."
FYI . .. .
Darrin
Cool. Good to know. +T
-
run chkdsk on your hard drive. This was the 1st symptom I had before a crash.
-
well, the plot thickens. I tried to start pulling wav masters off my main drive and I ran an md5 on each folder. When I copy/move the folder the md5 gets mostly bad verifications. This includes going to another drive or trying to burn to dvd. I had a similar problem going to a server, so I'm guessing that is the same issue. Running checkdisk now. I'm on my lappy typing this message.
In addition to my main 120gb drive I have a 2nd that is 160gb, and a 200gb external. I've been lazy about backing up to dvd and have tons of stuff on all drives. I just tried to move something from my external to my 2nd internal and the md5 didn't check out. Does this still sound like my maind drive?
I updated my windows this morning as well before I posted this thread.
Couple of things that may be relavant.
I seem to remember starting a defrag about a month ago and having to stop it before it was done. I don't know that I ever did one since. Will a partial defrag screw things up?
I replaced my motherboard but used the same chip. I put the drive in and it booted up no problem. I just loaded all the motherboard specific drivers and didn't reload windows. Could that be screwing me up?
Matt
-
try encoding at a lower compression rate. Im guessing you use over 6 (which is a waste of your time But thats awhole other thread.)
Im betting thats your problem. Seems to bite alot of folks over here.
edit for fat fingers
-
I updated my windows this morning as well before I posted this thread.
can you elaborate on this comment Matt? windows update, i presume?
also, have you tried uninstalling/reinstall the FLAC or Shorten program?
just a thought.
-
I tried that, but typically I do use setting 8...never had any problems till recently.
Matt
-
I updated my windows this morning as well before I posted this thread.
can you elaborate on this comment Matt? windows update, i presume?
also, have you tried uninstalling/reinstall the FLAC or Shorten program?
just a thought.
yes to all...
I've also been having problems with mozilla crashing on me too... I'm starting to think this is my drive... checkdisk is done, I'm going to start a defrag now and see if that helps.
Matt
-
another thought: try running flac/shorten in safe mode and see how it responds. could help eliminate whether spyware/virus is causing you problems.
F8 on bootup
-
another thought: try running flac/shorten in safe mode and see how it responds. could help eliminate whether spyware/virus is causing you problems.
F8 on bootup
T to you sir... hadn't thought about that one yet... actually t's to the thread guys! Thanks for all the help.
Matt
-
Matt,
Sounds like bad memory to me. When copying data around it gets copied via your RAM. When you're running flac w/verify, it's verifying as it goes, so I'd guess whatever is being verified is still in cache... as soon as it hits RAM you lose.
The error message, "bad crc" would tend to indicate it is NOT a hard drive failing... you'd expect to see disk I/O errors if that were the case. And your disc would be thrashing a lot, making weird noises. I just had a hard drive fail on me earlier this week, it made some horrible noises and brought the computer down with it (but hey, the swapfile was on that drive :D ).
It COULD also be your cpu overheating. Make sure the fan isn't obstructed, and the heatsink is seated properly. Add more heatsink compound (the white goop) is needed. After I removed my dead hard drive, I managed to jam my fan, and had exactly this sort of failure.
Are you overclocking? If so, try running at the correct speed. Are you memory timings set too fast? Be more conservative. Run a memory tester (memtest86?).
Good luck, and hope you figure it out soon!
best regards,
stephen
Edit to add: you can safely abort defrag while its running, that's not the problem.
-
Matt,
Sounds like bad memory to me. When copying data around it gets copied via your RAM. When you're running flac w/verify, it's verifying as it goes, so I'd guess whatever is being verified is still in cache... as soon as it hits RAM you lose.
The error message, "bad crc" would tend to indicate it is NOT a hard drive failing... you'd expect to see disk I/O errors if that were the case. And your disc would be thrashing a lot, making weird noises. I just had a hard drive fail on me earlier this week, it made some horrible noises and brought the computer down with it (but hey, the swapfile was on that drive :D ).
It COULD also be your cpu overheating. Make sure the fan isn't obstructed, and the heatsink is seated properly. Add more heatsink compound (the white goop) is needed. After I removed my dead hard drive, I managed to jam my fan, and had exactly this sort of failure.
Are you overclocking? If so, try running at the correct speed. Are you memory timings set too fast? Be more conservative. Run a memory tester (memtest86?).
Good luck, and hope you figure it out soon!
best regards,
stephen
Edit to add: you can safely abort defrag while its running, that's not the problem.
good stuff stephen... I know nothing of the things you just mentioned. I didn't add any white goop when I moved the cpu from one motherboard to the other. I don't know much about computer building,I was sort of winging it. I have no idea how to overclock so unless that defaulted I'm not doing that. Memory or cpu cooling are starting to sound like good things to check at this point... memory came from one motherboard to the other with the cpu... I will check that stuff out. Thanks and +T
Matt
-
does anyone know how memtest86 works? so far I have a blue screen with a title and nothing hapening for about 15 minutes no indication that it is working and it won't take any input from my keyboard.
Matt
*edit* I just hit my 2000th post... I feel so old!
-
did you put it on a floppy ???
im running it now for the hell of it, hopefully i'll be right back ;)
-
I don't have a floppy on my machine so I did it as a bootable cd.
let me know how it goes, but I'm running late for a little bbq smoker event that I deperately need after all this shit today. I may not be back till tomorrow.
Thanks bean
Matt
-
scratch that... just threw it in the lappy and it is working fine ???
-
I have a PC which has a DMA data corruption problem when writing to disks on the internal IDE ports. Basically, if you copy 50 gigs of flac to it, approx 4 or 5 files will be corrupted. I couldn't fix it via drivers, etc, and ended up installing a promise ATA controller and not using the built-in ports.
Silent data corruption. Fear it!
It takes a Lot more time to completely verify files after they are copied. I use rsync under linux. I don't know how good the windows version is.
-
Sorry, just have to say this.... And people wonder why I recommend Macs over Wintel computers...
Wayne
-
FWIW, I would be willing to bet you have some bad ram in this case too. I have had that bite me on an older machine. Ram upgrade, everything was fine.
-
mmmatt - I've had the EXACT same problems as you. I had 2 hard drives in my machine, one with my program files (C:) and another huge drive that I was using just for storage (D:) - I did a reformat of C:, and all of the sudden, many of my flac files would not decode - both on D: and on CDs and DVDs I burned. I've given up on trying to fix them, and have sworn off flac becuase of it.
I think my issue was a drive corruption - after the reformat, I made a new directory on D: that was all of the .wavs from old flacs that I could actually decode, and they were all being saved. The resultant .wavs coud be played in several audio programs. I got Nero, and tried to burn the .wavs onto a DVD, and the directory was corrupt. When I clicked on the directory, it gave me a message that the directory was incomplete or corrupt - and this was not flacss, but wavs. I don't know if my reformat of c: caused the corruption on d: - unlikely b/c they are physcally different drives.... or if it was just coincidence....
I've had several discussions with freelunch and a couple w/ MikeW, and done hella searches online, and nothing has worked. I searched the exact error text online, and most resolutions were achieved through re-downloading the original flac files, indicating a partial copy of the flac. I've since started using a prog called "copy large files" to move the wavs around, per advice from Freelunch about losing data when copying - he uses rsync, but I suck at command line stuff, so I needed something with a nice interface.
It really sucks, but I'd bet money that your drive is dying. Exactly the same issues I had. Unfortunately, I'm not sure if you'll be able to recover your FLACs. As much as I liked flac, I'm done with it. At least with some other file format, the audio would be somewhat salvagable.
sorry bout your luck dude.... I was there... I'm still there.... just had to come to grips with losing all kinds of stuff I considered rather valuable. 'twas my first hd failure.
-
you guys are really making me nervous. i have 100's of dvd's in flac. i had this happen to me w/ shn, and swore those off. fuck.
-
A drive error has nothing to do with the algorithm. Regardless of how what format your data is stored in, if you have a bad sector you generally have lost the data in it. As mentioned earlier, FLAC can decode through errors, so if the remainder of the file is still intact, then it's mostly recoverable. With a compressed file, one sector of compressed data holds more information than a sector of uncompressed data, so you'll lose more of the original wave if it's compressed.
With the move away from physical tapes, I urge people to realize that digital data essentially doesn't exist unless it exists in three distinct places. I generally have two synchronized hard drives, a binder of DVDs and I regularly FTP shows to a friend who also keeps them around. The latter is a good backup method: find a trusted friend who likes listening to the stuff you tape and tell them you'll give them a copy of everything you record under the condition that they keep it.
Using multiple backups has saved my ass in the past. Before I had a pair of drives, I had one die. "No problem," I thought, "I'll just use the backup CDs." Unfortunately for me, one of the CDs wasn't good; I could only pull about half of the files off it. I was able to track down a copy from someone I'd traded with nearly a year before. I learned my lesson then and instituted my current system. To save time from FTP transfers, my buddy recently mailed me a 120 GB drive. I filled it and will send it back this week.
-
With the move away from physical tapes, I urge people to realize that digital data essentially doesn't exist unless it exists in three distinct places. I generally have two synchronized hard drives, a binder of DVDs and I regularly FTP shows to a friend who also keeps them around. The latter is a good backup method: find a trusted friend who likes listening to the stuff you tape and tell them you'll give them a copy of everything you record under the condition that they keep it.
i also burnm the master flac sets untracked onto cd-r
but this is def the ebst way to make sure there are copies around
and also, usually, if a flac cd/dvd doesnt wanna copy verify, in my cases, it ahs only been 1-2 tracks and theyre usually easily found and easy to get 1-2 tracks rather than losing a whole show
-
I've posted this before but it should be repeated... I monitor the health of all my drives using the built in SMART capability.
When I get a new drive, I immediately run a 'short' test on it (2 minutes) and then a 'long' test (about an hour). The drive tests itself - they are pretty smart these days. A drive I recently bought couldn't even pass the short test.
http://smartmontools.sourceforge.net
-
A drive error has nothing to do with the algorithm. Regardless of how what format your data is stored in, if you have a bad sector you generally have lost the data in it. As mentioned earlier, FLAC can decode through errors, so if the remainder of the file is still intact, then it's mostly recoverable. With a compressed file, one sector of compressed data holds more information than a sector of uncompressed data, so you'll lose more of the original wave if it's compressed.
Well, I have no problem defering to your expertise... but from what I've been told, if a FLAC header doesn't accurately describe the FLAC data, then you are hosed. I don't think the same can be said about WAV. Also, with my FLACs being hosed, I've lost everything. You can't lose much more of a file than "all".
-
FWIW, I have one of his bad flacs and it has a bad header. I really can't say what condition the rest of the file is in.. But when I try to extract with -F, flac pukes right on the header and says it isn't a flac, game over. I could dig into the flac header format and try and reconstruct the header (maybe copy a header from a diff file, etc) but it would be a bitch. WAV headers are very simple, I could reconstruct one in my sleep. Though we should try the -F on some of the other problem flacs.
-
I hadn't considered an error in the header. Your best bet would be to reconstruct the header by looking at a known good header in a hex editor and extrapolating from that using the FLAC specification. So long as the data is there, I wouldn't call it lost. You might want to shoot an email to one of the FLAC developers asking how to go about recovery. I've gotten replies from Josh Coalson in the past.
-
Good news!!!! I just replaced my ram and it seems as though all is working fine! Please keep the info going on the flac fix though... I've got some multitrac masters that are crap unless I can figure out the issue.
Thanks to all that helped and also to bean. ha! T's to the thread (aka "a round for the house on me")
Matt
-
Oh wow, errors in the header = not good :(
Here is my backup strategy: I take the master .wav and compress to .flac, I master the show = a bunch more flac files. I wait til I have a bunch of shows - enough to fill a dvd. After I burn a dvd, I do a file-by-file comparison (comparing every byte in every file) between the newly-burned dvd and what's on my hard drive. Until it's archived on optical media, I keep a copy on my jb3, a copy on my laptop and a copy on my desktop machine. I've got a ton of dvds in a storage wallet, which I'm looking at now with some concern!
I'm curious.. what backup scheme do the professionals use, e.g. archive.org... Diana?
Edit: great news with the new ram... backatcha!
best regards,
stephen
-
Oh wow, errors in the header = not good :(
Here is my backup strategy: I take the master .wav and compress to .flac, I master the show = a bunch more flac files. I wait til I have a bunch of shows - enough to fill a dvd. After I burn a dvd, I do a file-by-file comparison (comparing every byte in every file) between the newly-burned dvd and what's on my hard drive. Until it's archived on optical media, I keep a copy on my jb3, a copy on my laptop and a copy on my desktop machine. I've got a ton of dvds in a storage wallet, which I'm looking at now with some concern!
I'm curious.. what backup scheme do the professionals use, e.g. archive.org... Diana?
Edit: great news with the new ram... backatcha!
best regards,
stephen
I recall her saying that they have a few levels of backup on HD. The archive started as a test for long term hd storage... hopefully she'll chime in and say that all better.
I flac, and md5 all my masters, as well as my finished files and then copy to dvd and put into a binder... that is how I originally figured this little issue out. My md5 didn't clear on one of my dvd backups, and I was like... oops. Then I went back to the original data on the hd and the md5 cleared. I then tested the flac files and some were bunk. Then I was OK for awhile and then the shit hit the fan.
Matt
-
I'm curious.. what backup scheme do the professionals use, e.g. archive.org... Diana?
One of the things done is put things on DVD-RAM (basically you're using it as a DVD disc). DVD-RAM is the only format that has built-in error checking into the recording scheme (it's also one of the reasons why Zaxcom hasn't moved away from DVD-RAM as the only non-hard drive media the Deva writes to). We probably need to come up with some good practice backup techniques for all of this since a lot more people are moving to non-tape solutions for recording. Then we need to put those in the FAQ section.
Wayne
-
interesting...
-
I'm curious.. what backup scheme do the professionals use, e.g. archive.org... Diana?
Edit: great news with the new ram... backatcha!
best regards,
stephen
I recall her saying that they have a few levels of backup on HD. The archive started as a test for long term hd storage... hopefully she'll chime in and say that all better.
I'm not a techie so I can't give any specifics, sorry. I gather IA has at least 1 Bay Area copy of each thing plus a European copy (Amsterdam?). The moving images and the audio collections are currently in 2 different types of filing/storage methods. Later the audio will be the same as movies.
When there has been a catastrophic loss of a chunk of LMA stuff (as happened at the end of June), I think it's because items were moving off the upload disk onto a failing public d/l disk, and being lost before they had a chance to be mirrored.
-
thanks diana!