Topic: DV codecs and colorspace

[dklein]

Figured we may as well work this out in the open in case anyone else runs into the same stuff.  Starting with this previous post from a thread hijack:

thanks for the props, dklein.

i'm all about the video geekin'.     been fascinated by it ever since VCD mpgs became popular online, and went from learning all those programs to some of the more complex DVD-related programs out now. (haven't started with Scenarist yet, but i have DVDMaestro down very welll, and most people are really impressed with the stuff i pull off using it.)

i never really used VirtualDub that much, except to extract WAV files from AVI files, in the past... until this year. because i do most editing in Premiere (which needs Type 2 DV .AVI files, as opposed to Type 1), i downloaded the Panasonic DV codec so i could check out VirtualDub. (mostly because i was considering deinterlacing some footage for a DVD project.) BIG mistake. simply put, the Panasonic DV codec sucks a fat one. my source was extremely clean (read: NO grain, only clear crisp picture), and after using VirtualDub with the Panasonic DV codec, the exported AVI files had friggin vertical lines. and not small ones either. probably 12 or so of them starting from the left side of the picture all the way to the right side.

i did a triple take when i watched the test MPEG-2/DVD on my TV, and started wigging out. i pulled out the DV masters and popped them in - looked perfectly fine. went back on the computer, did an MPEG-2 encoding of an original captured DV .AVI, looked excellent... no vertical lines. so then i did some googling about the panasonic dv codec, and got some hits. apparently i'm not the only person who encountered this problem.

but in my searching and reading about the panasonic dv codec, i read up on others as well. read an excellent review with screen caps on multiple codecs and the reviewer showed how the Canopus dv codec rocks the cazbah. unfortunately, Canopus dv codec is generally only available if you get an expensive piece of hardware by them... unless you snag ProCoder 2.

i grabbed it, and i have to say, i may have to start using it (just to re-render the footage). i did a side-by-side test of encoded MPEG-2 video: captured DV. AVI file > MPEG-2 vs. re-rendered with the Canopus dv codec > MPEG-2. the Canopus blew away the straight up video... absolutely brilliant colors (made the original capture look bland, halfway desaturated, and very dull). also, the blacks are nice and rich just as it is on DV-shot footage. (ever watch a DV tape directly on TV vs. a DVD made from the same DV source? the DVD will have duller blacks... it goes back tot he 16-235 colors thing you mentioned.)

i'd definitely say CCE is the way to go (FrameServing or not) for encoding. MainConcept is okay, but i read a detailed review with screen caps regarding ProCoder, TMPGEnc, CCE, and MainConcept. out of those 4, MainConcept was clearly the worst quality of all the encoders. granted, it's not expensive compared to the others (especially CCE or ProCoder), but i could not believe the difference in quality from the screen caps. FYI, ProCoder came in a very close 2nd and TMPGEnc was almost as good as ProCoder, IMO. but i'm not a big fan of TMPGEnc (the MPEG encoder) - it always ran freaking slow on my machines. and right now, CCE 2.70 (even without FrameServing) is a screaming beast for me. my machine will do a 6-pass encoding of 2 hours of footage, which gives me great quality encodes (NEVER a single bit of blocky pixelation or bluriness during fast-motion scenes) in 3 hours.

i could do the 2-pass encoding with MainConcept of the same 2 hours of footage in 1 hour, but i try to keep the quality... and more passes really helps a lot, especially if you're encoding something at bitrates lower than 6000-7000 or so.

sorry for rambling on and on. i don't know enough video geeks, obviously. ;-)

Shoot us the link to that comparison site.

[dklein]

As far as my experiments go, I've tested out the Panasonic codec, the Sony 2.23, the MS codec and the internal one in Vegas.  Made some very interesting discoveries.  Getting a good picture seems to be a matter of matching up the codec with proper encoding parameters.

Codecs come in 2 flavors - squished and non-squished (I'm trying to avoid the word compressed).  In TV land we need to finish with something squished because TVs can't use the full range of colors like a PC.  If not, it looks over-saturated with burned out whites and loss of detail in the black.  So this exercise is mostly about where and how many times the squishing occurs.

4 Codecs I tested
Panasonic and Microsoft - both output 0-255, RGB24 - non-squished
Sony and Vegas internal (MC?) output RGB24, 16-235 (or at least a relatively squished color range) - squished

I was able to get all of them looking good with the right settings but the squishiness is the key to getting the colors right. 

If you have a squished codec and you want to frameserve to CCE, you must frameserve in RGB so that you can tell CCE to leave the luminance alone (set 0-255).  You can't feed it YUY2 or it seems to squish and look bad.

If you have a non-squished codec and you want to frameserve to CCE, you can feed it YUY2.  If you feed it RGB, you need to set luminance levels to 16-235 (and let CCE squish the levels).

In terms of quality, I think the Sony codec (the standalone, not vegas)  may be a touch funky on the reds - it is subtle.  I thought the Panasonic looked as good as the Vegas internal stuff when all was set right.  I didn't spend too much time trying to get the Microsoft codec looking good as 3rd party codecs are recommended, even by Microsoft.  When you get the squish factor wrong the difference is not so subtle.  It's either way too zingy or totally hazed over.

As far as encoders go, it appears that:
Vegas encoder (Main Concept) wants a squishy source (like it's internal codec or the Sony codec).
CCE is configurable for RGB sources, but if you're feeding it YUY2 it needs a non-squishy codec.
I haven't used TMPGEnc in a bit - it has similar parameters around the color space (the setting is under MPEG settings, quantize matrix, Output YUV data...)  When the box is checked, I believe it's the 'no touch' option, similar to CCE's 0-255 range.

Any feedback from more knowledgable sources is welcome! I'd like to put Procoder thru the same tests.

I'll try and post some screen grabs.

[dklein]

1. DV>CCE (16-235 luminance range), panasonic codec correct
2. DV>Vegas (internal codec) correct
3. DV>Vegas (panasonic codec) too zingy -never got squished
4. DV>Vegas (internal codec)>frameserved to CCE in YUY2 format washed out - double squished - by the codec and then CCE
5. DV>Vegas (internal codec)>frameserved to CCE in RGB32 format (luminance set to 0-255) correct - same as above but avoided the 2nd squish by framerserving in RGB and setting luminance to 0-255

[dklein]

1. DV>CCE (16-235 luminance), sony codec washed out
2. DV>CCE (0-255 luminance), sony codec correct (same as above, corrected by using proper luminance range in CCE)
3. DV>Vegas (sony codec) correct
4. DV>CCE (0-255 luminance), panasonic codec too zingy - this is the same as the 1st one in the above post, but with the luminance range set wrong

[dklein]

whew...4 solo posts.  OK - I don't think things are as obvious in the stills as they are in video.  The washed out ones really are, and there's a lot more pixellization in the dark parts of the video.  With the zingy ones, you tend to lose any detail in the black curtain and the drummer's shirt isn't distinguishable.  The skin tones show some of the differences.  Also, the whites just look overexposed.

By the way, all the screen shots are from a 9k 1 pass so it's not really an encoder test.  Once I have a handle on the codec thing I'll compare encoders again, but in my limited experience, all 3 are quite capable.  I couldn't tell the difference b/w Main Concept and CCE when both were at 5k 2 pass.

[hyperplane]

the thread i referred to about tests was regarding MPEG encoders. however, that is a side-related item (because i think the spirit of this thread is to basically try to solidify how to go from video source -> DVD with optimal quality). here's the thread:

http://www.videohelp.com/forum/viewtopic.php?t=235665&postdays=0&postorder=asc&highlight=cce%20procoder&start=0

of course, there are lots and lots of comments by people (some who have applicable knowledge, and some who don't know jack).

i think the screen shots at the beginning of the thread speak a lot by themselves. (you have to scroll up and down a good bit to compare one encoder vs. the DV source file... or just save the pics to your HD and use an image viewer to quickly flip back and forth).

on 1 pass encodings, you can probably get very similar results with MainConcept, TMPG, CCE, etc. but CCE really shines when you put it up (and past) 3 passes + VAF file (which is really 4 passes all told).

as for the screen shots you posted, dklein - there are definitely differences. look at the guitar on the far left... it's easy to spot some difference in the sharpness and coloring in that part of the picture.

*supposedly* from what i've read, say you capture video to your hard drive (camcorder > Firewire > scenalyzer live/Premiere/whatever) as a Microsoft DV .AVI file. again, this is just what i read, the originally captured DV file is *supposedly* un-changed from the original file. so a straight capture > encoding to MPEG obviously is better, since you're eliminating variables (read: going through another DV conversion and possibly screwing up the settings or using a bad/less desirable codec).

more useless info based on my google research:

- Avid, Canopus, Sony DV codecs are all supposedly "good." problem with the Avid codec is, from what i can tell, it captures the video as OMG instead of AVI (type 1 or 2). so i don't know if vegas or premiere would open/import an OMG file, and i think those two are generally the more popular video editing programs on the pc platform.

- ever notice how you can record (off tv or film) a video with your DV cam, then hook up the camcorder to your TV and it has the rich black colors. then encode to MPEG and the blacks aren't the same at all.... i assume this goes back to the typical 16-235 color range for NTSC. still, a bit of a pain.


also, dklein - my panasonic tests were NOWHERE near the quality of the screen shots you posted. i tried every setting i could alter on it (whilst in VDub) - and every single test came out very grainy with those friggin vertical lines (which were more noticeable in lower lit scenes, but still unacceptable even in more brightly lit scenes).

i think you may agree with me that video > DVD is a bit more intricate of a process than doing a DAT -> CDR transfer properly. ;-)

[dklein]

The reason I used the 9k 1 pass was to try and eliminate encoding quality differences as much as possible and just show the codec.  I figured that 9k 1 pass is about the top - because the whole picture is encoded at the highest allowable bit rate.  But I guess multiple passes can still optimize that bit rate -just didn't think of it earlier.

As far as the firewire transfers gp, the codec is irrelavent as the data is straight from the camera.  But as soon as you want to do something with that file (watch it, edit it, convert it to mpeg2) you need a codec to read it.

[hyperplane]

very, very interesting dklein.

i read in one of the doom9 stickies that it's pretty much always a good idea to set the colorspace in CCE to 0-255. i have to wonder, though, since the NTSC spec is basically limited to 16-235, is there going to be a noticeable difference when watching a DVD encoded (with setting at 16-235) on a DVD player/TV setup? i may have to run some tests on this to see if i can tell any difference.

[firmdragon]

very, very interesting dklein.

i read in one of the doom9 stickies that it's pretty much always a good idea to set the colorspace in CCE to 0-255. i have to wonder, though, since the NTSC spec is basically limited to 16-235, is there going to be a noticeable difference when watching a DVD encoded (with setting at 16-235) on a DVD player/TV setup? i may have to run some tests on this to see if i can tell any difference.

comparing 16-25 and 0-255 on your tv is fairly obvious which one you want to pick. (0-255)

[dklein]

You don't necessarily want 0-255.  For TV, you want 16-235 because that represents the TV's full range under the NTSC spec.  The 0-255 is for more capable pc monitors.  What ends up happening if you go with values of 0-255 on a TV is that you clip your blacks and whites because the TV can't cover the range.

So at some point in the process, you must make an adjustment in order for the full range of colors to be visible on a TV.  When you tell CCE to go 0-255, all you're really doing is saying "don't mess with my colors".  When you specify 16-235, you're saying "adjust my colors to fit the tv range".  And that's only if you feed it RGB source.  If you feed it YUV then the conversion does not take place and you'll need to check your codec.

It is important to know the behaviour of both pieces - your codec and mpeg2 encoder are doing because you want to do this once and only once.  You can always run some tests and the difference is obvious.  It's either zingy, in the middle, or hazy and light.

[firmdragon]

hmm interesting post dklien +T for the info.  i wasn't aware of that to that extent.  my main point i guess was that if you were to compare the 2 ranges you'd know what to use right away.

on a side note.  i'm using screen caps from my videos for a collage of sorts.  when i extract them the caps are pretty dark when compared to on tv.  is there a photoshop filter or something i can use to make them look brighter/more like on tv?

[dklein]

go Image, Adjust, Levels and then change the range to 16-235 (it will be at 0-255)
I think somewhere there's a PC>TV adjustment but I couldn't find it.  I hate Photoshop 

[firmdragon]

t+hanks!!!!!!