There has been much discussion among HD DVD and BD enthusiast sites like AVS Forum about whether banding seen in some titles is due to the encoding process or the source. I can't speak for other codecs, but for VC-1, our codec does a great job of being accurate to the source, and so banding seen in HD bitrates is almost always in the source.
The good folks at the Orange Open Movie Project produced Elephant's Dream, a Creative Commons licensed 3D animated short, and have made a PNG sequence available of it. I took those samples, assembled them into a single .AVI file, and encoded it up using the current released version of the Windows Media Video 9 Advanced Profile codec (our brand for our implementation of VC-1). Here are the basic settings I used in Windows Media Encoder:
We of course are going to dive into some registry keys to make this look good. Here are the settings I used in WMV PowerToy:
So, why these settings?
DQuant, short for Differential Quantization. DQuant varies how much different macroblocks (16x16 blocks) of the image are encoded, applying relatively less compression to smoother parts of the image. This reduces banding and blocking. We apply this to I and P frames only (B-frames are based on the I and P frames on either side, and are able to maintain the smoothness and texture of them, so leaving DQuant off for them saves bits). And we use the Regular mode, as it's the appropriate default. If we saw banding, we could override the codec and tell it to take flat areas down to QP (Quantization Parameter, with lower meaning less compression) of 2 or even 1. But that takes bits away from other parts of the image, and wasn't useful in this case.
Perceptual Option - Adaptive Deadzone 1 lets the codec tune its quantization table to provide more detail in smoother areas and a coarser representation in highly textured areas (with so much detail the lower precision isn't noticeable). Adaptive Deadzone isn't for all content, but works nicely for this kind of content with fine texture. It works well with film grain too.
In-Loop Filter applies a deblocking filter to the video in-loop, meaning that future frames are predicted based on the deblocked versions of previous frames. This means that any errors (even subtle non-visible ones) in one frame aren't propagated forward to future frames, and lets us lower the data rate needed for a particular quality level. I always have this on.
Overlap Filter applies further deblocking between block boundaries. It can risk softening the image some, but at this high resolution, a little softening of individual pixels isn't a big deal. Most of the clip looked great without Overlap, but it definitely helped in the cable section when the camera is rotating. All those sharp curved lines with global camera motion are a real killer! I only apply Overlap filtering when the video is getting a little blocking.
Motion Search Level- Fixed True Chroma tells the codec to do a full search for how motion changes in the chroma (also known as U and V or Cb and Cr) as well as the luma channel (also called Y). For highly colorful content like this, often there are bigger changes in color than there are in contrast, and chroma search will be able to compression that content in both higher quality and higher efficiency. I could have used the Adaptive mode and it would have looked nearly as good and been a little faster, but it was a pretty short clip.
Motion Search Range - Adaptive tells the codec to determine the optimal motion search range to use for each frame. Using a small motion search range is faster and more efficient when there isn't much motion, but bigger ranges are much more efficient when there are big changes. Bear in mind that this needs to compensate for the motion between two P-frames, since B-frames aren't referenced. In analyzing the output file, the motion search range alternated between 64/32 and 128/64 depending on how much motion there was. Note that we always search more left/right instead of up/down since that's where most motion goes, and also the screen is rectangular, so there simply isn't as much room for big vertical motion.
Motion Match Method - Adaptive tells the codec to pick the optimal choice between SAD (Sum of Absolute Differences) and Hadamard motion matching for different parts of the screen. Hard to explain in examples, but Adaptive is the highest quality mode, with SAD the fastest (and default).
B-Frame Number: 1. This turns on B-frames. B-frames have three big advantages
- They improve encoding efficiency. 1 B-frame is most optimal in VC-1, unlike MPEG-2, due to our superior I and P-frame encoding. The 1 B-frame is just a hint, though. The codec may insert extra B-frames in some cases (look below) or turn them off for particular sequences like fades to/from black.
- They improve random access. Since no frame references a B-frame, B-frames can be ignored when decoding previous frames in a GOP to get to a particular frame
- They enable "BI" frames. A BI frame is a B-frame that is all Intra-coded; technically it compresses like an I-frame, but an I-frame that no other frame references. These enables content that lasts for a single frame like a flash to get turned into a BI frame, leaving the frame after the flash to reference the frame before it. This provides a lot of the value of multiple reference frames without the decoder complexity in MIPS or memory.
As far as the original goal, It demonstrates nicely that there is banding in the output when there's banding in the source, and that there isn't banding in the output when it isn't in the source. Anyone is free to compare my file with the PNG sequence I used for source.
As a secondary goal, the clip does a great job of showing off the media playback power of the new Xbox 360 Spring '07 update. Although this clip hardly makes it break a sweat - I've been able to do a full 1920x1080 60i with 30 Mbps peaks on it without a dropped frame! I'll try to get some of those samples uploaded as well. I also get good playback on my quad AMD box and my new Core 2 Duo laptop, but my aging XP dual Xeon 3.4 box couldn't quite keep up.
UPDATE: And I've finally figured out how to get microsoft.com to host the video file. Here's the Elephant's Dream 10-25 file for download!