Topic: Taper's Linux HOWTO

[Brian Skalinder]

Translating between int and float creates noise because rounding errors are introduced during the storage format conversion.  Try reading a 2G file into audacity and then writing it out to another file name.   You will rarely get an exact duplicate.

I have yet to figure out how to get audacity to stay in 24-bit int mode even though there is a setting that seems to be intended for that purpose.

Sorry, I'm not making myself clear.  My main qualifying question regarding this statement...

Most Linux audio tools don't do 24 bit well.  They quietly convert to 32 bit and back to 24, introducing noise.

...was:  At what point does this conversion to 32-bit and back to 24-bit take place?  Under what circumstances?  If we're talking during a simple file save, for example, then yeah - I understand how rounding errors would prove problematic.  But if the transformation from 24 > 32-bit and back only takes place during editing, then my follow-up question still stands:  is the suggestion that <1> using 32-bits for more precise math during editing and then dithering back to 24-bits noisier than <2> performing the editing with less precision in 24-bit and without dither?

Not sure if that's any clearer...   

[chase]

while this is a nice resource, unfortunately, it doesnt tackle the main problem with Linux recording...configuring the interface driver.

even on the ALSA page, many cards that are "supported" arent, at least the digital I/O portion of the interface isnt supported.
take the M-Audio PCI cards for example...sure, the Envy24 chip is supported, if you only use the analog RCA I/O. it's been the case for at least the last 6 years and that is IF you manage to get ALSA to load the driver in the first place (I've tried with 2.6.x kernels and I still cant get ALSA to load correctly...)

those of you who use Linux to transfer digital recordings, what interface are you using?

I have an M-Audio Revolution 7.1 which uses the Envy24 chipset and it has been detected and loaded properly in SuSE 9.2, 9.3, 10.0, and ubuntu.  They all seem to suffer from the same mixer problem though.  Whenever it loads, the left channel is always muted, although this can be easily fixed by opening your mixer of choice and bumping the levels to take it off mute.

[Brian Skalinder]

Translating between int and float creates noise because rounding errors are introduced during the storage format conversion.  Try reading a 2G file into audacity and then writing it out to another file name.   You will rarely get an exact duplicate.

Bingo!  [snip]

Okay, I think I get this now - in the process of simply reading and then re-writing a 24-bit WAV file, with no editing, Audacity is not bit-transparent since it (behind the scenes) converts the 24-bit file to 32-bits and then back again to 24.

This all got me curious, so I just installed and fired up Audacity 1.2.4 on Windows 2000 SP4.  FWIW, after changing the output format to 24-bit (from the default of 32bf), opening a 24-bit file in Audacity and exporting (saving) to 24-bit WAV has proven bit-transparent every time I've done it (about half a dozen, so far).  I wonder if the latest version has solved the problems you've experienced, FL, or if something fundamentally different is happening behind the scenes in the Win version v. the Linux version.

At any rate, FL, sounds like you've banged your head against that wall plenty in the past, and I don't blame you for bailing - I, too, would much rather record and listen than dork around with computer config stuff constantly!

[homer420]

In response to the other suggestion, aplay/arecord don't have any design features to prevent audio interruptions due to other system activity. ecasound does.  For that reason, the alsa team recommends against using aplay for live recording. ecasound is also extremely well supported by the author.

Correct, arecord/aplay do not have this capability built-in, however the Linux OS does provide ways to accomplish this.  You can change a processes scheduling priority by envoking it using the nice command.  In the 2.6 kernel there's multiple I/O schedulers available, for which the deadline scheduler provides near real-time behavior.  And since arecord/aplay are command line apps you can boot into say runlevel 2, which would prevent most system services from starting/running.  You could even customize runlevel 4 to be a "live recording" runlevel which only starts services absolutely necessary to do recording.

FWIW I have been using arecord for DAT to HD transfers for over 2 years now and have yet to run into any interupt or scheduling related problems.  Admittedly I have yet to use it for live recording purposes, however I don't see how this would really change things in terms of the kernel scheduler and interupts ...

[it-goes-to-eleven]

Correct, arecord/aplay do not have this capability built-in, however the Linux OS does provide ways to accomplish this.  You can change a processes scheduling priority by envoking it using the nice command.  In the 2.6 kernel there's multiple I/O schedulers available, for which the deadline scheduler provides near real-time behavior.  And since arecord/aplay are command line apps you can boot into say runlevel 2, which would prevent most system services from starting/running.  You could even customize runlevel 4 to be a "live recording" runlevel which only starts services absolutely necessary to do recording.

Another reason I avoided arecord was lack of support. I recall that the authors strongly suggested using a refined tool and that they wouldn't waste time supporting these types of issues.  Kai's support of ecasound is Amazing. I've written apps that use alsa to do data acquisition and I really wanted something more refined than that for live recording.

I did do a lot of work to optimize my linux laptop rig - power consumption, scheduler testing, low latency kernel, text only.  I had it setup to automatically start recording when powered on (if the minime was present).  It worked quite well. But I still didn't like the hassle of a laptop at shows.  I thought about building a dedicated headless recorder..

At some point I reminded myself that all the tweaking and optimization - exactly what you mention above - had nothing to do with recording live music. And it was a lot too close to work. So I wimped out, bought a 722 and started doing 15 shows a month 

[Lil Kim Jong-Il]

Okay, I think I get this now - in the process of simply reading and then re-writing a 24-bit WAV file, with no editing, Audacity is not bit-transparent since it (behind the scenes) converts the 24-bit file to 32-bits and then back again to 24.

This all got me curious, so I just installed and fired up Audacity 1.2.4 on Windows 2000 SP4.  FWIW, after changing the output format to 24-bit (from the default of 32bf), opening a 24-bit file in Audacity and exporting (saving) to 24-bit WAV has proven bit-transparent every time I've done it (about half a dozen, so far).  I wonder if the latest version has solved the problems you've experienced, FL, or if something fundamentally different is happening behind the scenes in the Win version v. the Linux version.

I've done tests in which Audacity read in a large file and wrote out an identical copy.  But in many other cases it didn't.   Someone posted about this a while back on their Windows machine and when I ran tests on my Linux machine, I got perfect copies so I made a wise crack about windows.   Now I've proven that I don't always get perfect copies and it's the same revision of Audacity I was using for the successful tests.  Sometimes I get a perfect copy until well into the file and then there are small errors.   

If you want to use audacity to resample and dither, then it's a decent program.  It just isn't capable of non-destructive editing.  From what I've seen of the traffic on the audacity user's list, we're so far beyond the intended user segment that I don't think they will do anything to fix it.  I couldn't even get anyone to respond to my posts about this issue.   I am running 1.2.3 so maybe I'll try 1.2.4 and report back.  I have files handy that have demonstrated this problem.


A very annoying thing about these errors is that what I'm assuming are rounding errors are not simply off by 1 as I would expect.  Sometimes the values are off by as much as 4.   I never bothered to use the file offset to calculate the byte position, I just always assumed it was the LSB.   If the errors were in the middle or MSB I would think that would show in the display and I never saw anything when I examined the waveform or listened.

I keep attributing these errors to rounding however after a beers worth of thinking I'm not sure that is correct.  The format for 32-float is: 1 bit sign indication, 8 bit exponent and 23 bit mantissa.  So a 24 bit integer value should convert to float and back with no issue.  Where you should get into rounding errors is when trying to represent integer values greater than 24-bits or real numbers that cannot be accurately represented in a 23-bit mantissa.  So while I tend to blame the errors on the 24int/32fl conversions, I'm not really sure that is the culprit.   Anyone have thoughts about this?  I haven't done any real CS stuff in a long time.

[it-goes-to-eleven]

Kai thought the simple 24>32>24 case should be accurate but I've found cases in ecasound where it isn't.  He suggested using libsndfile (an easy work-around in ecasound). It was more accurate but still not perfect. He attributed it to rounding/truncation.

Some applications will use libsndfile for doing conversions and audio file I/O.  Or.. If libsndfile is not available at compile time, they'll use their own internal routines (which have less functionality).

I'm also on the audacity list.. It isn't very active and people have trouble getting help. I agree that the typical user is far removed from our requirements.

[homer420]

I did do a lot of work to optimize my linux laptop rig - power consumption, scheduler testing, low latency kernel, text only.  I had it setup to automatically start recording when powered on (if the minime was present).  It worked quite well. But I still didn't like the hassle of a laptop at shows.  I thought about building a dedicated headless recorder..

At some point I reminded myself that all the tweaking and optimization - exactly what you mention above - had nothing to do with recording live music. And it was a lot too close to work. So I wimped out, bought a 722 and started doing 15 shows a month 

Completely understood.  I started going down the same path with the laptop but decided quickly having a laptop at a crowded bar gig wasn't for me. And thus I still record to DAT for much the same reasons

However I needed to break my dependence on windoze and M$.  So I set off to find linux apps which provided the same (simple) functionality I used for transferring DAT's to HD, splitting tracks, and burning CD's.  For high-end audio work linux is definitely lagging behind.  But it certainly does the trick for the basic functions I required, which is really what the Taper's Linux HOWTO is geared towards - tapers looking to move to linux for doing the standard transfer-split-burn type activities ...

Thanks again for yours and everybody elses feedback on this thread.  It's great to be part of this community 

[pigiron]

Not officially however pigiron noted earlier in the thread that he's hacked 24-bit ALSA support into wavbreaker.  And thus I've requested he send me a patch so I can add it to my packages an begin playing around w/ it.  If all seems good I'll work w/ the author to get it integrated into future versions.

pigiron - feel free to just send me the source for your hacked version and I'll generate the patch ...

Sorry... I have a habit of disappearing sometimes... if you find me, show me the way home 

Patch sent via email.

Damn interesting discussion go'n on here... that's what I like about having the source code... you can actually see what the sucker's doing... and if ya don't like it, fire up the compiler.

[pigiron]

forgot to mention...

i run a headless linux jukebox for playback... so i hacked flac123 to handle 24 bits if anyone's interested in a lightweight command line flac player that's seamless between tracks... works great for the stuff we record.

[it-goes-to-eleven]

I'd like to check out your flac123 mods.. It is a bit frustrating that a lot of these tools won't do 24, let alone 24/96..  I also use headless Linux for playback (even gave a headless linux music server to my folks one year ago.. still going strong!). I'm currently using the Slimdevices squeezebox but need to get 24/96 support in the new listening room. I have an idea on how to "easily" add the support to squeezebox.

[MattD]

I have an idea on how to "easily" add the support to squeezebox.

I'd be interested in hearing this.

[mfisch]

forgot to mention...

i run a headless linux jukebox for playback... so i hacked flac123 to handle 24 bits if anyone's interested in a lightweight command line flac player that's seamless between tracks... works great for the stuff we record.

Let me know when it will compile on PPC =)

[pigiron]

I'd like to check out your flac123 mods.. It is a bit frustrating that a lot of these tools won't do 24, let alone 24/96..  I also use headless Linux for playback (even gave a headless linux music server to my folks one year ago.. still going strong!). I'm currently using the Slimdevices squeezebox but need to get 24/96 support in the new listening room. I have an idea on how to "easily" add the support to squeezebox.

yep, know the frustration, but don't forget that this stuff's coming from a bunch of overworked/hung-over nerds doing it for fun/love... but I'll tell ya... if you get that SB working and you'll be a hero for a lot of folks!

The flac123 patch is so short, I'll try to attach it to this reply.

Here's the link to the original source code:  http://flac-tools.sourceforge.net/

now... undocumented (of course) usage... just go into a directory containing a bunch of flacs and enter "flac123 *flac" and off it goes... Ctl-c to skip to the next file... Ctl-c two or twelve quick times to prematurely exit. It has a "remote" mode for a frontend app... but that's a whole other thread.

And to mfisch...

Let me know when it will compile on PPC =)

brother!!!  I sling a bunch of PPC code... don't have access to a Mac... but I can get it compiling on one of these http://www-03.ibm.com/servers/eserver/pseries/hardware/highend/ ... will that do? 

seriously tho... i see that flac123's been tested on Mac OS 10.3.9 .... so if ya wanna give it a spin...


Sorry Dave... didn't mean to hijack the thread... now back to our regularly scheduled program, already in progress...

[mfisch]

And to mfisch...

Let me know when it will compile on PPC =)

brother!!!  I sling a bunch of PPC code... don't have access to a Mac... but I can get it compiling on one of these http://www-03.ibm.com/servers/eserver/pseries/hardware/highend/ ... will that do? 

seriously tho... i see that flac123's been tested on Mac OS 10.3.9 .... so if ya wanna give it a spin...


Sorry Dave... didn't mean to hijack the thread... now back to our regularly scheduled program, already in progress...

[/quote]

I'll give that a whack on Tiger this weekend ... I havnt compiled anything on mac yet =)
guh, you guys know theres no gui flac player for mac? maybe I'll learn xcode this weekend too.