Topic: combining 16 bit/44.1k audio with HD video

[mdogbucket]

I posted the same question in the computer forum but have not gotten any answers in the last two days, so I thought this might be a better place.


My friend and I have been recording live music using stereo mics into a Korg MR-1000 DSD recorder.  Now we are video taping as well using one of the new Sony HD AVC-HD camcorders.  The problem is how to most efficiently and easily put the high res audio together with the HD video.

Is anybody doing the same thing?  I would like to do it two ways.

1. combining and synching the HD video with our audio at 24bit/96k or even 16bit/44.1.
2. dumbing down the HD video to something like DVD quality and combining and synching the audio

We were thinking of buying Sony Vegas Platinum pro.  Any thoughts?

[Ken Kaiser]

I had done a bit of this in older versions of Premiere Elements, and it seemed to work OK with 16 or 24 bit / 44.1Khz, but it needed to "render" the audio.  I don't know what kind of impact that had on the audio, but other than being slow it sounded OK.

Now whenever I record for video though, I set the recorder to 16-bit/48Khz, same as the video, and it drops right in the timeline with no issues.

[mdogbucket]

I'm fine with dropping in 16 bit/48k audio, but I will be down-converting from the original files recorded at DSD 1 bit/5.6mHz.

I guess I am really wondering if Sony Vegas Platinum pro lets you replace the crappy sounding audio that your HD camera picked up with the high quality stereo I capture with my mics and Korg recorder?

[Ken Kaiser]

I don't have Vegas, I've only used Adobe, but from everything I've heard I have to believe you can no problem.  I suppose to be 100% sure though wait for someone who has/uses it to respond.

[tedyun]

I've used older versions of Sony Vegas, and Final Cut Pro, and the newer versions should be similar.

When you open video footage in the timeline, the video track and audio tracks are separated. You can then overlay your alternate audio in a separate track and "slide" it around to sync it with the original audio.

I haven't found a more accurate way to do this other than by manual. Generally, you can zoom in on the timeline and try to visually line up peaks, such as a drum beat or vocals, then let your ears do the rest.

Once you have everything lined up, you can delete the original audio. I think in FCP, you had to "unlink" the original audio from the video. I can't remember if Vegas does the same thing.

The problem I had when I did this is that when I rendered the final file for DVD output, the audio would unsync with the video.

I'm fine with dropping in 16 bit/48k audio, but I will be down-converting from the original files recorded at DSD 1 bit/5.6mHz.

I guess I am really wondering if Sony Vegas Platinum pro lets you replace the crappy sounding audio that your HD camera picked up with the high quality stereo I capture with my mics and Korg recorder?

[markr041]

There is a perfect solution: Plural Eyes software, which is a plug-in for either Vegas Pro or FCP. Automatic sync in either. I have used this in Vegas Pro and it works great. I tape concerts in HD and record the audio seprately at 24-bit, 48khz. The final DVD (or AVCHD DVD) contains the video and the 24-bit sound uncompressed and untransformed, without the inferior camera-based audio 9which is used to sync the video to the separate audio track.

http://www.singularsoftware.com/autosync/

[willndmb]

i tried plural eyes with fcp and it didn't work at all
after a nukmber of emails to tech support and their "fix" i gave up
it was a sweet concept but like i said never worked right

[markr041]

Luckily there is a trial version to try without cost. What was special about your task that it did not work for you but works for thousands of others?

[nomotrouble]

I've been doing this all year on my mac computer, using final cut and imovie. I record audio on an iriver using at853 mics, and video with a sony hcr hc9 mini dv camcorder. Upload the video, extract the audio, line up the new audio, size it with final cut, then send it to idvd or make a quicktime file. My audio is consistently 9 frames per hour too long, so i adjust it first, then fine tune if necessary, which it rarely requires. Sometimes there are anomalies in the video which require splitting it and making sure it all lines up still.

[rsimms3]

I use Vegas, but I don't know of any HD format that will render with uncompressed audio into a final HD video format.  I use a Sanyo HD1000 camera which records to .MP4.  I render to AVC first, and then import to Vegas.  From there I go to the beginning of the video to sync both audio sources.  I then go to to the end to compress/expand the audio to match the camera audio since the two don't run exactly the same in their timing.  Then, mute your camera audio, and render to whatever format you want; your camera audio is now replaced.  If you want to go to DVD, render your video in the same way and then render the audio alone into the original format but this render will be your adjusted audio synced to your video.  Load the video into your favorite encoder/dvd creator and replace the audio with your uncompressed rendered audio.  Now you can encode your DVD with uncompressed audio, skipping one generation of loss or encode using uncompressed/synced audio. 

Sample having used this process:
http://www.youtube.com/watch?v=H5nOJuSGpYc

[mdogbucket]

Sample having used this process:
http://www.youtube.com/watch?v=H5nOJuSGpYc

That's a really nice looking video.  What format & resolution did you down-convert the high res audio and video to?

[rsimms3]

My original Sanyo Video was 720p 60fps MP4/AAC.  I ran that through PowerDirector 7 to utilize the Steady filter to reduce shake in the video, rendering to 720p 29.97fps HDV MPEG.  I imported that into Vegas 9.0 to sync the external audio, fade the beginning/end, and explorted again to 720p 29.97 HDV M2T.  I then uploaded that file to YouTube.  In using the Steady filter, you loose some of your sharpness and border as that filter cuts the sides and expands the picture so it can reduce the shakeness.  The more filter you use, the more it crops on the sides and you loose sharpness.  The Sanyo camera has the advantage of being small and portable, but disadvantage of being light weight and routinely has a lot of shake when handheld.  I don't use the built in camera filter to reduce shake as sometimes I get tearing at the bottom of the picture or some other unwanted side effets.  I have looked at upgrading to the next model up, still a couple years old at this point, but haven't felt the need since I shoot very little and I can get by with this camera.  My only real issue is battery life, but fortunately it takes 5v in and can be run on external battery packs so if I am running open, I have plenty of battery options.

[stevetoney]

I don't think anyone's mentioned it yet, but video needs 48khz, so first you need to convert your 44.1 sampling frequency up to 48.

I do it a little bit differently than others.  I first strip out the audio from the video using software the does this (I use DVD Lab Pro for DVD mastering, which includes this capability).  Then I use an audio workstation software, such as Audition to synch up the two audio sources by paralleling them.  I parallel the sources using stretch/shrink and then listening along the timeline of the audio and making whatever adjustments may be necessary to get the two audio sources perfectly paralleled. You don't want to touch the audio that you stripped out because that's of course perfectly synched-up with the video.  Once you have the better audio perfectly parallel with the stripped audio, then obviously it will synch perfectly with your video.  At this point, you can trash the original audio and replace it with the new audio your video mastering software (Again, I used DVD Lab Pro, but many people use Vegas).  Note that you might have to delete some portions of the new audio to compensate for any video drop-outs that may have occurred during video recording.

This method is a little more cumbersome and time consuming than simply stripping the old audio out and using Vegas to stretch or shrink the new audio to fit, but I like my method because a) I never have a problem with synching after I complete this process, and b) my method compensates for any drop outs that may exist in the video.  The simple stretch/shrink method won't compensate for drop-outs.

[rsimms3]

Is there an easy way to sync with Audition?  The only way I have found is very cumbersome by putting both tracks into the Multitrack editor and then shrink/expand one track.  I have Audition 1.5 and it won't shrink/expand in real time so it is almost a guessing game to adjusting as the program has to process the file before showing you the adjustment whereas with Vegas it will do it in real time so I find it much more exacting. 

I usually fill in gaps in the video with still images using a photo slide show.  If it is just a high number of frame drops in a particular section, I just sync as best as possible with the shrink/expand and go from there.  I have been happy with my resulting external audio syncs to this point. 

[stevetoney]

Is there an easy way to sync with Audition?  The only way I have found is very cumbersome by putting both tracks into the Multitrack editor and then shrink/expand one track.  I have Audition 1.5 and it won't shrink/expand in real time so it is almost a guessing game to adjusting as the program has to process the file before showing you the adjustment whereas with Vegas it will do it in real time so I find it much more exacting. 

I usually fill in gaps in the video with still images using a photo slide show.  If it is just a high number of frame drops in a particular section, I just sync as best as possible with the shrink/expand and go from there.  I have been happy with my resulting external audio syncs to this point.

I haven't used Vegas much for synching audio, so I'm not really sure what you mean when you say 'real time'.  Perhaps I should use it more and then I might abandon using the technique I've mentioned above.

Regardless, I think that both methods essentially do the same thing...you use the stretch/shrink function to line up the original audio against the replacement audio so that the replacement ends up being completely synched/parallel with the original.  Then once the replacement audio is totally paralleled from start to finish, you discard the original audio and remux the video with the replacement. 

Regardless of whether you're using Vegas or Audition, I've found that if the video isn't a continuous video that runs from start to finish but has cuts or drop outs, it makes the process quite a bit more time consuming, but not really all that difficult.  You just have to locate the specific cuts/drop outs and figure out how to change the replacement audio to match.  I'm not sure if that's easier to do in Vegas or Audition, but apparently either tool will accomplish the task.

[rsimms3]

In Audition 1.5 the way I use it, when you shrink or stretch the audio, you don't get to preview the change before the processing of the file.  In Vegas, you use the mouse and hold the Ctrl key to shrink or stretch the audio and can see the end result as you do it instead of having to wait for Audition to process the file and then show you the change you made.  I don't know how Audition is set up, if that is something meant to utilize less resources by processing the file, then changing it or what, but I much prefer Vegas for syncing.  Also, that shrink/stretch feature in Vegas isn't really advertised in the instructions or in general.  I think I did a web search on expanding audio and stumbled across it.

[stevetoney]

^ OK, understand what you mean now.  FWIW, it's never really been an issue for me in Audition either because you key the stretch/shrink to the amount of time that the whole audio needs to be adjusted and then enter the stretch/shrink ratio...it's always dead nuts for me.  However, it sounds like you do this sorta visually with Vegas, so you don't have to mess with ratio's at all.  Just align the two files up and do a sort of drag and drop method...where the new audio file just kinda stretches and shrinks like slinky.  That would definitely be easier for sure.

[rsimms3]

Yep, exactly.  Math was never my strong suit....throw in ratios and I am toast.

[boyacrobat]

if using vegas
go to your audio master track and click the plug inns/effects tab
by default sony inc the compessor and eq with a roll off in the chain.
remove all plug inns in the chain.

render an uncompressed audio track this way.

very important to your final mix.
the sony compressor is harsh in the chain when you compare the differance
do it for dynamics sake

g

[Shadow_7]

I guess I'm the cheapskate of the bunch.  I do this exact thing also with a Korg MR-1000.  I record DSD audio, convert to 24/96 and do some other edits(EQ/Amp/HL/???).  I use audiogate (obviously), audacity, sox, faad and ffmpeg.  With a sanyo FH1 for now and plans for a more pro-ish cam someday.

# The workflow....

Capture audio - Korg MR-1000 - DSD 5.6MHz
Capture video - Sanyo FH1 - 1080p60

# extract camcorder audio from video...

ffmpeg -vn -sn -i video.ext -acodec copy -y audio_only.aac

# convert camcorder audio to wav...

faad -o audio_only.wav audio_only.aac

# convert korg audio to wav...

audiogate

# identify a sync point (a bit trial and error)...

audacity korg_audio_24_at_96.wav
--- File -> import -> audio --- audio_only.wav

# extract the needed audio segment from the korg audio...

sox korg_audio.wav extract_audio.wav speed 1.00011 trim 00:00:05.555 00:01:00.000

Check to see how close my guesses are and adjust to the 1/1000th (music video so it matters).  1/10th or so is probably good enough for most folks.  Repeat ID / Extract steps until perfect.  Usually about 3 tries, 1st to be on the right second alignment.  Zooming in on waveforms to align the right peak(s).  Highlight opposing peaks then zoom and minus one zoom and do the math, adjust repeat.  When matched the above sox command with the right offset and duration gives the desired segment of audio.  Edit in EQ and such and convert to resulting format/resolution.  In audacity and export selection new_audio.wav

Then join the new audio with the video element...

ffmpeg -an -i video.ext -sameq -r 30000/1001 -s hd720 -pix_fmt yuv420p -f yuv4mpegpipe - | ffmpeg -f yuv4mpegpipe -i - -qscale 1 -i new_audio.wav -y new_video_with_external_audio.mpg

A bit of a time suck and a round about way to do it, but it gets the job done.  For those of us with more time than money anyway.  A lot of this is from memory, so there might be errors in my syntax's.  And some of it simplified.  HD sources are a bit new, so you've got to run the latest versions of a few software parts, which might require compiling from source, so not for everyone. 

I've got most of my regular conversions scripted.  i.e. for DVD versions I run a script it generates audio only, video only, and the dvd video with camcorder audio.  I swap the audio only part and rerun the same script and out comes a DVD version with external audio.  No need to re-encode the video segment a second time and all that jazz.  Hopefully that's not too much to digest.

The speed option for sox is because there's about a 1/2 maybe 1/5th of a second difference (per hour) between the speed of the audio on the camcorder and the speed of the audio on the field recorder.  Negligible for clips < 5 minutes.  But drummers at 60p in excess of 10 minutes..... noticeable.  It also helps ID the sync point better if you're not using the exact same hand clap / drum hit / whatever each time.

A bit techy I know.  A major time suck.  But mostly functional for me anyway.

[digitallive]

I posted the same question in the computer forum but have not gotten any answers in the last two days, so I thought this might be a better place.


My friend and I have been recording live music using stereo mics into a Korg MR-1000 DSD recorder.  Now we are video taping as well using one of the new Sony HD AVC-HD camcorders.  The problem is how to most efficiently and easily put the high res audio together with the HD video.

Is anybody doing the same thing?  I would like to do it two ways.

1. combining and synching the HD video with our audio at 24bit/96k or even 16bit/44.1.
2. dumbing down the HD video to something like DVD quality and combining and synching the audio

We were thinking of buying Sony Vegas Platinum pro.  Any thoughts?

kinda like what the other guy was saying, you can just dump both your V/A from cam and your HD audio into vegas, then synch the audio to your video (making sure you keep the video starting at 00:00:00:00) and then render the new audio as a WAV file (either delete the A/V file from cam or mute the audio from it). check various spots in the video to make sure it synchs up fine all the way to the end.

[Shadow_7]

Basically you want your audio segment to start before and end after the video segment.  Extract a segment of said audio at the right offset of equal length to the video segment.  And either add it to the video with the existing audio, or swap out the video's audio for the external one.  You may or may not need to compensate for clock drift since you're syncing two independent devices without worldclock / timecode / .  For segments less than five minutes it's close enough with my setup.  But most of my recordings are in that ten to twenty minute lengths, so I compensate.  It also makes it easier to use any sync point as the sync point doesn't change (much) if you choose a different one with each trial and error attempt.  There are software packages out there that will do this guessing for you to find the offset that matches.  In the absense of a hand clap or drum hit it might be the only sane way to handle that guess work.  Fortunately what I record has claps and drums.

http://www.youtube.com/watch?v=6bV4RRTJRRU

One I did a couple of weekends ago for a friend.  You can see the Korg and mics in the video.

[Matt Quinn]

I'm not sure about the DSD part, but I've developed a way to do this with a Flip Ultra HD cam that basically could not be any easier. Zero stretching etc required. The only software required is a multitrack audio editor of your choice, and a program called DVD Flick.

In my case, I use Ableton Live as my editor. Simply drag & drop the video file to a track, then drag the hi-res audio to the next track. Find a sync point (easiest way IMO is to pick the drummer counting off the tune, or banter between tunes). I will pan one source hard left, and one hard right, then move the 2 clips around until they are dead in sync. Live is particularly awesome for this as you can turn on a very narrow grid & slide your clips along it until you find the right spot.

Once your audio is lined up, select the portion that acompanys the video, and export the AUDIO ONLY as a wav - 16/44.1 or 16/48. You should have a wav that is the exact same length as the video.


Open up DVD Flick. Drag your video into the main window, and it will add it to the project. The program is pretty self explanatory, but the 2 things that may be a bit tricky:

Go into the 'audio' tab, remove the audio track that is listed (it is the camera's native audio track). Then add the new audio file you exported from your editor. After you add the new audio, you have to go right back into it, click 'Edit' ,then there's a little box at the bottom that says 'Ignore audio delay'. CHECK THIS BOX. Repat these steps for each video file. Hit 'Create DVD' in DVD Flick, and you are done.

That's it. I have done this flawlessly 4 times now. There is no drift between audio & video even with ~ 1 hour files.

[BayTaynt3d]

Vegas has virtually unlimited tracks, so that's never an issue. I regularly drop loads of video and full multitrack audio on the timeline no problem.

As for the real-time thing, Vegas is insane for that, some people familiar with other editors can't believe their eyes when they see that. Basically, you can be playing the audio (hearing it real time off the timeline), WHILE you drag or expand/contract audio or video events/tracks. So, basically, you can zoom way into the timeline, and move the video event a frame forward or backward (or move the audio for that matter), and literally listen to how the changes affect the sound as you do it. It can be helpful to just loop on a small region as you're doing things like that. It makes it SOOOO much easier to manually sync.

Also, I use Plural Eyes to great success in Vegas, that tool ROCKS, especially for someone like myself shooting on a 12-minute-limited Canon 5D2 where I end up with loads of clips to be synced. There are a few options you can use to help it make sync if it struggles, and I have yet to not be able to sync something with those tweaks.

Finally, I find all versions of Vegas 9x to be somewhat unstable (at least in my workflow, including the new 9d), but Vegas 8c is rock solid stable for me on a Win 7 i7 box. Another thing is that it's helpful to have your audio at 48hz to sync to hopefully the ambient track of your video (which will undoubtedly be at 48). As far as "replacing" the video's audio, no need in Vegas, just mute that track once you have sync and leave it right there in case you need it later.