Topic: 24 bit upload to LMA???

[Gordon]

I've uploaded only one 24 bit show since updating my flac.    the email I have with archive is my old one I never check.  anyway I checked it yesterday and had 5+ of the same emails from archive saying md5 failed etc etc.  it was weeks ago when I uploaded the show and it went live immediately.


http://www.archive.org/details/dbt2007-01-10.ak40s.flac24


and as far as I can tell has stayed live.  I haven't done anything or responded to the emails cause I just got them.     

so in my case it seemed the show was up and available for download even though they were emailing me about the "failed md5's".  strange.


EDIT:  while the show is "live" you get a error when trying to download the files.  this is complete bullshit!

[dactylus]

hmm, weird. ive upped prolly 75+ 24-bit flacs onto the LMA and have never had a single problem. I have been using 1.1.3b and 1.1.4b w/ out ANY problems!

Bean - not weird at all.  as long as you are using a version of FLAC v1.1.x, you'll be fine with the LMA.  the problem comes with those of us using v1.2.1.  in the latest version of FLAC (v1.2.1), 24 bit encoding has been changed a bit (which greatly improves the compression ratio).  However, the older version of FLAC (which the LMA still uses) cannot decode and verify 24 bit FLACs encoded in v1.2.1.

as a work around to this, many tapers are holding off on the upgrading to v1.2.1 and are sticking with v1.1.4 to stay compatible with the LMA.  personally, I'm just holding off uploading any more 24 bit shows until the LMA updates to the latest version of FLAC.

I hope that this isn't too stupid a question - I use FLAC Frontend v1.7.1 the etree version and the latest version of Traders Little Helper.  How does that jive with FLAC (v1.2.1)?

[JasonSobel]

hmm, weird. ive upped prolly 75+ 24-bit flacs onto the LMA and have never had a single problem. I have been using 1.1.3b and 1.1.4b w/ out ANY problems!

Bean - not weird at all.  as long as you are using a version of FLAC v1.1.x, you'll be fine with the LMA.  the problem comes with those of us using v1.2.1.  in the latest version of FLAC (v1.2.1), 24 bit encoding has been changed a bit (which greatly improves the compression ratio).  However, the older version of FLAC (which the LMA still uses) cannot decode and verify 24 bit FLACs encoded in v1.2.1.

as a work around to this, many tapers are holding off on the upgrading to v1.2.1 and are sticking with v1.1.4 to stay compatible with the LMA.  personally, I'm just holding off uploading any more 24 bit shows until the LMA updates to the latest version of FLAC.

I hope that this isn't too stupid a question - I use FLAC Frontend v1.7.1 the etree version and the latest version of Traders Little Helper.  How does that jive with FLAC (v1.2.1)?

Both FLAC Frontend and Traders Little Helper are graphical user interfaces for the FLAC program.  with Frontend (I can't speak to TLH, as I don't use that program), when you hit the encode or decode button, a little DOS window comes up that says it is processing the files.  look in that DOS window, and it'll tell you which version of FLAC is being used.

[ljsurfer2002]

(Sorry for the cross post)

Okay,

I think a few things need to be clarified on how things work so that some of the energy can be correctly viewed and seen with an insight from behind the scenes in regards to 24-bit flacs, 1.2.X and 1.1.X flac, and 403-Forbidden on shows.

1.

is this problem really being addressed?  i now have three 24 bit recordings stuck in limbo because of it.  i keep getting multiple "md5 failure" auto emails sent to me from iasupport@archive.org regarding these seeds and to be honest, its getting quite annoying.  my multiple emails replies (which is what the directions in the email say to do) have gone unanswered.

http://www.archive.org/details/breakfast2007-09-22.bk4022.acm671.burke.flac24
http://www.archive.org/details/lf2007-10-02.bk4022.acm671.burke.flac24
http://www.archive.org/details/gpn2007-10-07.m20.jklabs.722.burke.flac24

It is not really being addressed. Sorry, but it is not. Please see below for more info. You will need to re-uploaded a 1.1.X version of the 24-bit flac files above if you want to stop getting the failure e-mails.

I have not seen your multiple e-mails. I hope that someone has gotten back to you. I do not get the iasupport@ box, but there is a human there that is supposed to be dealing with those e-mails. because your issue is one that is out of her hands, she probably has not been writing you back? Especially since the 1.2.X is a known problem with 24-bit files.

2.

I also now have a 24 bit show stuck in limbo with supposed "md5 failures".  it says that every single file "failed the flac decode test"...
http://www.archive.org/details/ymsb2007-10-18.m200.flac24

hopefully this issue gets resolved soon (although I realize that its probably not a priority for the understaffed archive team.  I don't mean to sound impatient with this post, I just thought I'd keep this topic up and active so more people are aware of it).

Again, the issue will not be resolved soon. It is not a priority and implementation LMA wide is too time-intensive at this present moment. Especially when 24-bit flac files encoded with 1.1.X work just fine with the system.

The solution fot his show as well as others like it (24-bit flac failing b/c it was encoded with 1.2.X) is to unfortunately re-encode the wav files to 24-bit flac with 1.1.X flac and upload. When it is done, please e-mail the admin's at etree@archive.org and let us know this is done. we will take down the 'bad' version of the show and leave up the new 'good' version.

I'm sorry but this is the only current alternative at this point and it is the one that will ensure your show is up for public downloads in a timely manner. Please don't wait for the LMA to upgrade to 1.2, it is a long way away.

3.

is there any way that a mod can override the md5 failures and allow the files to be downloaded anyway?  or is there anything else we can do?  or just hope that the archive engineers get to it soon and update their version of FLAC?

There may be a way, but the easiest / fastest way is the above. Re-encode the show and upload. That is the quickest thing you can do to ensure that the show gets downloadable asap. Again, the reality of the time table for the archive engineers updating the flac version is no time in the immediate future.

4.

the issue is still there.  Every couple of weeks I get an automated message telling me that my FLAC files are bad (and I always respond saying that it's a false positive, and that LMA needs to update their FLAC software).

Again, the quickest solution is to just re-encode and upload the 24-bit show with flac 1.1.X ... The automated messages about the flac files (yes, false positive, but still a true-positive in the eyes of the LMA system) will continue unless the older encoding is uploaded.

5.

Just encode your 24-bit WAVs in version 1.1.4 and you'll be fine.

Exactly. Please follow this advice, all.

6.

This going to be resolved? I just uploaded a show in 24bit and I am getting "MD5 Failures" for all my files. Encoded with 1.2.1b and I know they are legit.

The show appears to be available for download on my end, but clicking on a song gets a 403-FORBIDDEN page 
http://www.archive.org/details/mofro2008-02-01.mk4v.flac24

Looks like my 24bit files are up there and available for download now. 

did you do anything?  i guess I need to email a mod about mine.  shouldn't be issue though cause they should have that shit fixed.

Okay, It appears I need to clarify how things work on the LMA a little bit because there seems to be some confusion. Quick history lesson: Pre-May-2006 shows waited on a human (me; others) to 'approve' each and every live show before it 'went live'. That was difficult to keep up with, as well as the endless "how come you haven't approved my show yet!!?" e-mails. They got quite persistent. THe system changed so that shows went 'live' immediately so that they had a details page with links and everything. To ensure quality control, checks were still put in place to run after the show went live, to make sure the files all passed md5, passed internal flac check, had a .txt file describing the show, etc.

Now, the LMA uses FLAC version 1.1.2. Recently, FLAC 1.2.X has come out. 16-bit FLAC 1.2.X files uploaded to the LMA are NOT a problem. The deriver checks the files okay and mp3's / derivatives are created and all is good. BUT Flac 1.2.X 24-bit files ARE a problem. Our current system cannot handle them and it gives a file error and 'red line' tasks the 'derive.php' task in that show's history area (viewable to the uploader by clicking 'edit item' then 'item manager' then 'history'). These 'red lined' tasks are bad. when a show's derive.php task 'red-lines' that generated the 'md5 failure' e-mail (regardless of if it is an md5 failure, missing a .txt file, etc). In cases of 24-bit flac 1.2.X files, this e-mail also gets sent out.

Now, on to the issue of why some shows are still viewable and some are 'forbidden' ...

Take for example, the above linked mofro show:

http://www.archive.org/details/mofro2008-02-01.mk4v.flac24

If I clicked on the .txt file, it downloads & views:

http://ia341043.us.archive.org/2/items/mofro2008-02-01.mk4v.flac24/mofro2008-02-01.txt

BUT, if I looked at the history (if I was the uploader, I'd see it at:

http://www.archive.org/catalog.php?history=1&identifier=mofro2008-02-01.mk4v.flac24

I'd see a red-lined derive.php task. It failed the flac check b/c it was encoded with 1.2.X ... when shows fail, all the files in the show's folder become 'read only' and do not allow downloads. this creates the '403 forbidden' error that you see with files from shows that have failed the check, either legitly, or which are encoded from 1.2.X flac.

In the case of people getting multiple e-mails about the error, that occurs because I generate a list of ALL the shows that have red-line failures and I re-queue / re-run ALL of them once a week or two weeks. the idea is that in that span, people have gotten their auto-e-mail about a faliure, have gone in and checked out / fixed the show / checked in. But now to re-run the show, someone (like me) has to re-try the show's derive task. if they have finally put up that .txt file or fixed that one failing track, the derive.php task will now 'run clean' and will generate mp3's and be good to go. In the cases of 24-bit flac shows encoded with 1.2.X this error will ALWAYS fail and will ALWAYS auto-generate the e-mail. Sucks, but that's just the way it is right now. THe files will ALWAYS become '403 error forbidden' because the system cannot handle 1.2.X. Go back and encode it into what the archive can handle (FLac 1.1.X) and upload / import that as a fresh show and let us know. we ill take down the old one and you will not receive any auto-e-mails.

"shouldn't be issue though cause they should have that shit fixed." ... even though they should 'have that shit fixed', it will continue be an issue, whether it should be or not, because updating the flac version is not high on their priority list and is very difficult time-wise and they don't want to put the resources towards such a small problem.

Remember:

*don't sweat the small stuff

*if something is broken (LMA not being able to handle 24-bit 1.2.X), and another avenue correctly works currently (24-bit 1.1.X) just do it that way till the broken way is fixed

*the LMA is free and is a non-profit. The number of coding engineers for the ENTIRE archive.org project (which is about 400 times the size of the LMA and includes so many books, movies, audio, collections, and SO SO SO much more. see the links in the header) is about 2 humans. Maybe 3 humans on a good day.. If you want to donate money towards FLAC 1.2.X specifically being fixed, please donate here: http://www.archive.org/donate/index.php and you can specify what you want your money to go to.

If you do donate (I hope people do. Most people just complain and do not put their money where their mouth is) just remember to specify that you want it to go to the flac issue and the money will talk. Otherwise, see #5 above and move on with your life.

Tyler

[MattD]

Tyler,

I'm curious as to why the "fix" isn't a drop-in replacement of the LMA's FLAC binaries. It doesn't look like 1.2.x deprecated any functions.

[dactylus]

hmm, weird. ive upped prolly 75+ 24-bit flacs onto the LMA and have never had a single problem. I have been using 1.1.3b and 1.1.4b w/ out ANY problems!

I hope that this isn't too stupid a question - I use FLAC Frontend v1.7.1 the etree version and the latest version of Traders Little Helper.  How does that jive with FLAC (v1.2.1)?

Both FLAC Frontend and Traders Little Helper are graphical user interfaces for the FLAC program.  with Frontend (I can't speak to TLH, as I don't use that program), when you hit the encode or decode button, a little DOS window comes up that says it is processing the files.  look in that DOS window, and it'll tell you which version of FLAC is being used.

Gotcha - I was using the older version of FLAC and have now upgraded.  Thanks for the info.  +T

[ljsurfer2002]

Tyler,

I'm curious as to why the "fix" isn't a drop-in replacement of the LMA's FLAC binaries. It doesn't look like 1.2.x deprecated any functions.

I honestly don't know, I'm not an engineer. I only know that i've tried asking them to upgrade to 1.2 for the past 6 months that this topic has been around and basically they have said that they don't find it a priority. The engineer cycles are being spent in other areas of the archive.

I think it also has to do with the effort it would take to change the flac on the linux ubuntu servers that they have.

Again, I don't know the specifics why, I can just tell you that it isn't a priority in the near future.