Topic: SoX

[JasonSobel]

I've just recently started using SoX (v 14.3.1), mostly because I was curious about it, but I also have heard and read very good things about the sample rate conversion.  I can usually find my way around a command line app without too much difficulty, but I'm certainly no expert at the command line.

a few questions...

(1) My primary use of SoX will be to resample audio.  Output will always be 16 bit / 44.1 kHz.  Input will be 16/48 (from DAT transfers), 24/48, or 24/96 (from new recordings).  SoX came with a sample batch converter file.  This was the sample:

cd %~dp0
mkdir converted
FOR %%A IN (%*) DO sox %%A "converted/%%~nxA" rate -v 44100
pause

I modified it slightly when doing Tim's Hampton '03 recordings, and I changed it to this:

cd %~dp0
FOR %%A IN (%*) DO sox %%A "K:/transfers/ph2003-01-04.u89.flac16/%%~nxA" rate -v -s 44100
pause

in this example, I had already created the output directory.
my question is more about the resampling options.  the -v tells it to resample at "very high" quality.
I added the -s option, for "Steep filter (band-width = 99%)"

this is what the documentation says:

The −s ‘steep filter’ option changes resampling band-width from the default 95% (based on the
3dB point), to 99%. The −b option allows the band-width to be set to any value in the range
74−99.7 %, but note that band-width values greater than 99% are not recommended for normal use as they can cause excessive transient echo.

the docs also say:

By default, all algorithms have a ‘linear’ phase response; for ‘medium’, ‘high’ and ‘very high’, the phase response is configurable (see below).

ok, so my basic question is, are the -v -s options what I want for the best quality?  I don't need to specify -L, because the linear phase response is already the default for -v, yes?  Should I change any options? if so, what?



(2)  from what I read about dither, it seems that the default behavior for SoX is to apply dither to get down to 16 bit.  I assume leaving the default dither settings is fine?  is there anything that I would want to change?



(3)
some DATs that I transfer were originally recorded with pre-emphasis.
SoX can remove the pre-emphasis, with the deemph command.
the basic example they give is:

sox track1.wav track1-deemph.wav deemph

now, let's say I have a 16 bit / 48 kHz wav file, and I want to remove the pre-emphasis, and then resample to 44.1 kHz.  all together in one step, with dither applied just once, at the very end.  I think this is possible.  but how do I do it?


(4)
does SoX handle files over 2 gig?  Let's say I record a show at 24/96.  and the first set was 2.5 hours in length, and I have several files that are ~5 gigs in all (autosplit during the show by my recorder).  Can I use SoX to join the files together?  and then resample the 5 gig 24/96 file down to 16/44.1?  can I do this all in one step?  Can I also normalize each channel independently also at the same time?




Thanks for looking, and thanks for any and all help.

- Jason

[SmokinJoe]

I've used sox quite a bit for mix and bass roll off, and increase volume to 0, etc.

1) I don't normally use sox for dither/resampling.  It may do a fine job work.  My ears like "ssrc_hp" and that's what I've always used.
http://shibatch.sourceforge.net/ssrc/
I have a batch file... "ssrc_hp --bits 16 --rate 44100 --dither 3 --pdf 2 <24bitfile> <16bitfile>"

2) dither defaults... I don't have a good answer for this.  I generally use ssrc and resample/dither as shown.

3) I normally do conversions one at a time.  As a programmer, I try to think how I would apply algorithms, and I suspect that doing too many things at once might not yield optimum results.  Therefore I would do everything one at a time then dither/resample last.

4) For joining... I think at one point there was a 2gb limit, but I think newer versions are OK with it.  I normally use "shntool join", or combine the files in Audacity, do a level/fade, save to 24 bit, then dither/resample with ssrc.

The other big thing I use SOX for is to stretch or shrink a file by a tiny bit.... like when you do a AUD/SBD matrix mix from 2 recorders which aren't clocked.
sox <inputfile> <outputfile> speed 1.0000434   (where the 1.0000434 is a manually calculated adjustment ratio).

One time I had a bad mic cable, where one channel was wired up backwards, so my L/R channels were out of phase, and I had a bunch of stuff processed before I realized it.  I wrote a little script called "invert_right_channel" which did a "split L/R" with Ecasound, then flipped the one channel with "sox right.wav -v -1.0 inverted.right.wav"

~~~

edit: revised my previous statement because it sounded rather terse.  my apologies.

I didn't know there was a windows version of Sox.  I always used it in Linux.  I'll have to try the windows version.

[JasonSobel]

thanks for the replies, Joe.  I'll have to look into ssrc as well.

3) I normally do conversions one at a time.  As a programmer, I try to think how I would apply algorithms, and I suspect that doing too many things at once might not yield optimum results.  Therefore I would do everything one at a time then dither/resample last.

I think I'll have to read up more an the default dither behavior and learn to control it, because I'm pretty sure that by default, dither would be added after each command.  and when doing de-emphasis first, and then resample, I certainly don't want the audio to be dithered between those two steps, rather, have it added only once, at the end...

[it-goes-to-eleven]

24 bit support is a fairly recent addition to sox.  I have tried ssrc, though I ran into some serious memory consumption bugs that prevented me from really using it.  They've probably been fixed by now.

[JasonSobel]

In the months that have passed since I first posted this, I've spent a fair amount of time with SoX, and I've come up with several "common" effect chains that I tend to use.  I also enjoy the functionality and sound quality of SoX, so I thought I'd share here in case anyone else is interested.  These are all for the command line in Windows.  I'm not sure if Linux and Mac command line is exactly the same, but if they aren't, it should be very similar.  Obviously, if you're going to use any of these, replace the directory path and file names with the actual path/file name of your files.

These commands are what i would typically use for a new show that I've recorded in 24/96 or 24/48, and the files are split automatically by the recorder.

sox -V3 --combine concatenate "C:\music\transfers\input-file01.wav" "C:\music\transfers\input-file02.wav" "C:\music\transfers\input-file03.wav" -t wavpcm -s "C:\music\transfers\output-file.wav"

• Inputs multiple 24 bit files, combines them into a single 24 bit file.  No other processing. (The "wavpcm" parameter tells it to write a non-Microsoft standard 24-bit WAV file.  This is sort of like checking the "alternate 24-bit format" box in CD Wave.)

sox -V3 --combine concatenate "C:\music\transfers\input-file01.wav" "C:\music\transfers\input-file02.wav" "C:\music\transfers\input-file03.wav" -b 16 "C:\music\transfers\output-file.wav" rate -v -s 44100 dither -s

• Inputs multiple 24 bit files, combines them into a single file.
• Converts to 44.1 kHz (very high quality, steep filter)
• Applies noise-shaped dither
• Outputs 16 bit / 44.1 kHz file

sox -V3 --combine concatenate "C:\music\transfers\input-file01.wav" "C:\music\transfers\input-file02.wav" "C:\music\transfers\input-file03.wav" -b 16 "C:\music\transfers\output-file-norm.wav" gain -n -0.5 rate -v -s 44100 dither -s

• Inputs multiple 24 bit files, combines them into a single file.
• Normalizes (based on peak values) to -0.5 dBfs
• Converts to 44.1 kHz (very high quality, steep filter)
• Applies noise-shaped dither
• Outputs 16 bit / 44.1 kHz file

sox -V3 --combine concatenate "C:\music\transfers\input-file01.wav" "C:\music\transfers\input-file02.wav" "C:\music\transfers\input-file03.wav" -b 16 "C:\music\transfers\output-file-norm-peak-bal.wav" gain -en -0.5 rate -v -s 44100 dither -s

• Inputs multiple 24 bit files, combines them into a single file.
• Balances left/right channel volume such that both channel's highest peak is the same level
• Normalizes (based on peak values) to -0.5 dBfs
• Converts to 44.1 kHz (very high quality, steep filter)
• Applies noise-shaped dither
• Outputs 16 bit / 44.1 kHz file

sox -V3 --combine concatenate "C:\music\transfers\input-file01.wav" "C:\music\transfers\input-file02.wav" "C:\music\transfers\input-file03.wav" -b 16 "C:\music\transfers\output-file-norm-RMS-bal.wav" gain -Bn -0.5 rate -v -s 44100 dither -s

• Inputs multiple 24 bit files, combines them into a single file.
• Balances left/right channel volume such that both channel's average RMS level is the same (and prevents clipping)
• Normalizes (based on peak values) to -0.5 dBfs
• Converts to 44.1 kHz (very high quality, steep filter)
• Applies noise-shaped dither
• Outputs 16 bit / 44.1 kHz file

The next series of commands is stuff that I've used/will use when I transfer old DAT recordings, recorded at 48 khz, and recorded with pre-emphasis.

sox -V3 "C:\music\transfers\input-file01.wav" -b 16 "C:\music\transfers\output-file01-deemph-1644.wav" rate -v -s 44100 deemph dither -s

• Converts to 44.1 kHz
• Applies de-emphasis EQ curve (to remove the pre-emphasis
• Applies noise-shaped dither
• Outputs 16 bit / 44.1 kHz file

sox -V3 "C:\music\transfers\input-file01.wav" -b 16 "C:\music\transfers\output-file01-deemph-1644-norm.wav" rate -v -s 44100 deemph gain -n -0.5 dither -s

• Converts to 44.1 kHz
• Applies de-emphasis EQ curve (to remove the pre-emphasis
• Normalizes (based on peak values) to -0.5 dBfs
• Applies noise-shaped dither
• Outputs 16 bit / 44.1 kHz file

sox -V3 "C:\music\transfers\input-file01.wav" -b 16 "C:\music\transfers\output-file01-deemph-1644-peak-bal-norm.wav" rate -v -s 44100 deemph gain -en -0.5 dither -s

• Converts to 44.1 kHz
• Applies de-emphasis EQ curve (to remove the pre-emphasis
• Balances left/right channel volume such that both channel's highest peak is the same level
• Normalizes (based on peak values) to -0.5 dBfs
• Applies noise-shaped dither
• Outputs 16 bit / 44.1 kHz file

sox -V3 "C:\music\transfers\input-file01.wav" -b 16 "C:\music\transfers\output-file01-deemph-1644-RMS-bal-norm.wav" rate -v -s 44100 deemph gain -Bn -0.5 dither -s

• Converts to 44.1 kHz
• Applies de-emphasis EQ curve (to remove the pre-emphasis
• Balances left/right channel volume such that both channel's average RMS level is the same (and prevents clipping)
• Normalizes (based on peak values) to -0.5 dBfs
• Applies noise-shaped dither
• Outputs 16 bit / 44.1 kHz file

Hopefully someone else will get some use out of this, because I think SoX is a great little tool.

[trustthex]

I'm not afraid of the command line.     Interesting read & heads-up.  Also, thanks for the batchfile fodder.   



Looks like I've got some resampling to try.

[JasonSobel]

one more that I forgot to post earlier:


sox -V3 --combine concatenate "C:\music\transfers\input-file01.wav" "C:\music\transfers\input-file02.wav" "C:\music\transfers\input-file03.wav" -n stats
pause

• inputs multiple files, combines them, and then gives statistics, like peak level, average RMS level, etc, etc, for each channel
• does not output any new file

[rjp]

If you're running a Unix-type system, the following shell script will:

• Resample and dither all FLACs in your current directory to 16-bit/44.1 kHz, placing them into a flac16 subfolder
• Transfer tags to the resampled FLACs
• Transfer embedded cover art to the resampled FLACs

It can take advantage of multiple CPUs or cores, so it can process files in parallel; change the CPUS variable to suit your system.

Requirements:

• SoX, compiled to support FLAC format (most Linux distros should do this for you)
• FLAC and its libraries
• GNU Make

Keep in mind that all punctuation marks are significant in this code.

I call the script flacdown and keep it in a directory in my PATH, with the execute bit set.

Usage:
cd /path/to/your/flac/files
flacdown

If you don't have cover art in the FLACs, you will see "Error 1" messages, which can be safely ignored. After completion, the resampled files will be in the newly created flac16 subfolder.

Code: [Select]

#!/bin/bash

# On Solaris, change "/usr/bin/make" to "/usr/gnu/bin/make"
MAKE=/usr/bin/make

# Number of CPUs or cores available
CPUS=4

mkdir -p flac16

# Keep in mind that the Makefile is embedded in the shell script, between the two "EOM"s.
# The backslashes preceding the dollar signs prevent them from being interpreted by the shell.
$MAKE -j$CPUS -k -f - <<EOM
files = \$(wildcard *.flac)
targets = \$(files:.flac=.flac16)

%.flac16 : %.flac
        sox \$< -r 44100 -2 -t flac \$@ rate -m dither
        metaflac --export-tags-to=- \$< | metaflac --import-tags-from=- \$@
        metaflac --export-picture-to=\$<.jpg \$< >/dev/null 2>&1
        metaflac --import-picture-from=\$<.jpg \$@ >/dev/null 2>&1
        rm -f \$<.jpg

all: \$(targets)

.PHONY : all
EOM

mv *.flac16 flac16
cd flac16
for i in *.flac16
do
mv $i `basename $i .flac16`.flac
done


[JasonSobel]

for anyone else interested, SoX 14.3.2 was released a few weeks ago:
http://sox.sourceforge.net/

[dabbler]

I'm slowly learning to track/edit my own recordings, and SoX is a natural tool for a command-line junkie like myself.
Attached is a shell script I used to make my first (and so far only) recording
I've uploaded to LMA: http://archive.org/details/tg2012-04-01.dab.flac16

For future recordings, I'll have to figure out how to not apply volume gain to
crowd noise in between audio.  I don't think the "fade" effect can do it out-of-the box, so it may involve additional trimming + crossfade.

[rastasean]

Welcome, Eric!

I must say, nice job on making it all the way to this thread for being a new member here. Most people, and likely me, didn't make it past the ask the tapers section for many posts.

It looks like the script was not attached to the post, want to try again?

[Cheesecadet]

Wow...this thread is way beyond my basic computer comprehension. 

Just out of curiosity why not just use one of the many programs that allow you to enter all these edit features with just the click of a mouse instead of writing code?

Not being a smart ass here, I really am curious. 

[dabbler]

Cheesecadet: I never liked GUIs, I usually find websites annoying to use so it took me a long while to sign up for this one.  I probably drink too much coffee so I'm incapable of using a mouse correctly.  Scripting is more reproducible in case I want to make minor edits later on.

rastasean: I searched for tips on using SoX to get here :>.  Trying the attachment again (I suck at using using GUIs, actually)...

[JasonSobel]

Wow...this thread is way beyond my basic computer comprehension. 

Just out of curiosity why not just use one of the many programs that allow you to enter all these edit features with just the click of a mouse instead of writing code?

Not being a smart ass here, I really am curious.

I like SoX because it is easy to use (once you get the hang of it), and the resampling algorithm is really good.

also, BTW, SoX 14.4.0 was released a short while ago.

[trustthex]

I figured out how to downsample 24bit to 16bit from this thread, and have now saved my batch files.  I had been using r8brain, but could not in this instance...  This thread saved me likely minutes>hrs of headscratching.  Thanks Jason!