I gave a quick response in the comments that I wanted to expand into a proposed definition.

Resolution and frame rate

The infamous Table 3 of the ATSC specification lists 18 different resolution and frame rate combinations for digital broadcasting, But they drew a pretty clear line on what’s HD and what’s not: 1280x720 is HD, and anything less isn’t.

A more pedantic definition could be that HD should be at least 1920x1080p24 or 1280x720p60. And we’ll get there on the web before too long. But for now, I want the full 720p experience as a minimum bar. I can see two flavors of that which we can define as the ragged low end of web HD; anything below these are something else.

1280x528p24

For film source content, the widest aspect ratio in common use in 2.4:1. With 1280 wide, if we crop to the active image area and then round down to the next divisible-by-16 value for optimum compression, we’re left with 1280x528. For 1.85:1 movies, the equivalent is 1280x688

960x720p30

Lots of production codecs for HD aren’t square pixel. HDCAM is 1440x1080, and DVCPROHD is 1280x1080 or 960x720, even though both formats are always 16:9. So, I’m also inclined to allow 960x720 anamorphic for HD, but only in 30p. DVCPROHD’s 24p mode is 1280x1080p24.

Thusly…

If we calculate the pixels per second of the above, we get:

• 1280x528x24=16,220,160 pixels/sec
• 960x720x30=20,736,000 pixels/sec

So the anamorphic is still more pixels/sec than square pixel 2.4:1. We could arguably define “Web HD” as “at least 16M pixels/second” as well.

Quality

Hassan Wharton-Ali brought up another good point on the thread - HD should actually be HD quality. It can’t be a lousy, over-quantized encode using a suboptimally high resolution just so it can be called HD.

A good test is the video should look worse (due to less detail), not better (due to less artifacts), if encoded at a lower resolution at the same data rate. If reducing your frame size makes the video look better when scaled to the same size, then the frame size is too high!

Samples!

I know these are going to be too wide for my default blogging interface, but I need to stick an iframe somewhere in order to lock down the frame size exactly, due to how Silverlight Streaming works. Autoplay is turned off so your browser doesn’t automatically start two HD clips at once.

NOTE, yes I know in fact they still are doing Autoplay for some reason. I'm trying to fix that right now. Just make sure to pause the one you aren't watching. At least I got the size locked down (although still with too much padding...).

These are WMV files, but encoded with EEv2 SP1 using Smooth Streaming settings (so 1-pass CBR, 2 sec Closed GOP, and all that jazz). These are pretty basic encodes; I didn’t do anything tricky, as we WANT to see some artifacts as we’re defining these as the low bound of HD and below the low bound of HD. I do three versions of each clip

• True HD at 3 Mbps (64 Kbps audio, 2937 Kbps video)
• “Fake” HD at 1 Mbps (32 Kbps audio, 959 Kbps video)
• Good SD (640x360 for 16:9, 640x264 for 2.4:1) at 1 Mbps (48 Kbps audio, 943 Kbps video)

The SD is in there as an anchor to show that a lower resolution can actually look better at lower bitrates. Clearly the SD looks better most of the time, so 1 Mbps doesn’t count as HD. And those 16 Kbps of audio make a real difference with WMA 10 Pro, taking us from 32 to 44.1 KHz.

If you mouse the player, it’ll pop up controls (but without scaling the video – one of the nice features of the Black Glass template). To pick a particular clip, click on the icon that looks like poker chips (although upon reflection I note they’re actually film reels).

The Island: 1280x528p24

I haven’t used this clip in ages, but it’s always great to trot out as an edge case of hard encoding. It’s a Michael Bay joint, full of whip-pans, super-fast editing, frenetic motion, and film grain. Low bitrates can look fine for 80% of the frames in a clip, but fall apart for 20%. This clip has some good sequences where there’s multiple edits a second, so you get lots of chances to see the bandwidth stress. If only we had an encoder that could dynamically adjust frame size based on content complexity…

Lady Washington: 960x720p30

Just a short version of it this time, mainly because I’m tired of slow DSL uploads. The HD sizes are anamorphic, the SD is square pixel.

I figured this was as good a time as go through a hands-on project for Smooth Streaming.

The Source

We’ve had a few years of good fun with the Blender Foundation’s Elephant’s Dream clip (as I used here, here, and here). They have a new Creative Commons licensed title up called Big Buck Bunny, which is a lot brighter, and with a lot of nice chewy grass for high frequency detail. Elephant’s Dream is a great test clip for handling gradients and dark detail (It’s all about differential quantization), but Big Buck Bunny is arguably a better test for general compression quality, and also has some nice sky gradients as well.

I downloaded the stereo audio as a FLAC file, and converted that to a 44.1 KHz 16-bit WAV file with Audacity. I then downloaded source as a series of PNG files, and then used After Effects to combine the PNG sequence and the WAV that into a 1920x1080 Lagarith RGB AVI file. Since PNG and Lagarith are both lossless, and I didn’t do any scaling or color space correction, I didn’t do any preprocessing, just assembling the content into a file Expression Encoder 2 can consume. I’m a big fan of the quality of EEv2’s Super Sampling scaling mode, so prefer to use that instead of other tools where I have less control over scaling quality.

Setting up the Encode

Since we had other encodes up at SmoothHD.com, I wanted to follow the same data rate/frames size/GOP length parameters we’d already used. For the other encodes in the series. Specifically, the below, which give us seven bands from 2500 Kbps to 364 Kbps:

Tweaking

Smooth Streaming as a feature came in very hot for EEv2 SP1, and there wasn’t a lot of time to tune the presets.

There are a couple of safe tweaks I always recommend as previously mentioned:

• Adaptive Dead Zone=Conservative
• Overlap=On
• Scene Change Detection=On
• Search Range=Adaptive

Those help Smooth Streaming (and web-rate VC-1 in general) quality with minimal impact on encoding time.

For quality-over-speed projects, I also like to use

• Complexity=4
• Chroma Search=Full True Chroma
• Match Method=Adaptive

All together, that’ll increase encode time about 2-3x for a nice but not overwhelming quality difference.

And because this is animation content, I’ll also use

• B-Frame Number=2

In general, the less noise in the content and the closer the matches between frames, the more likely 2 B-Frames will be more efficient than 1. I’ve gone up to 4 B-frames for Camtasia screen recordings.

Charts!

One cool thing about Smooth Streaming encoding is that the independent chunks make analysis and graphing of the content a lot easier. We can analyze each chunk as an independent unit, and compare quality between bitrates and versions quite directly. Do note that there is some allowed variability in chunk size; we target for a 5 second buffer, so that any three chunks in a row should average around the same amount, but if a particular chunk is harder to encode than its neighbors, it can have a somewhat higher bitrate.

Here’s the .XLSX if you want to check out the source data:

I’ve encoded four versions of the content

• Default: the stock EEv2 SP1 Adaptive Streaming settings
• Fast: Adding the “safe tweaks” above
• HQ: Adding the quality-over-speed settings
• Insane: Taking HQ and adding "Insane” modes that add a lot more in encoding time than quality
  • Single-thread encoding
  • Complexity 5
  • Hadamard Motion Match

Insane is there to show what happens when you burn as much CPU as possible.

To show the impact of these different encoding settings on the video, I used an in-house command-line tool to dump a log file of the size, type, etcetera of each frame, and then did made the Excel spreadsheet linked above to see what interesting data I could find.

First off, let’s take a look at just a bunch of frames. Specifically, the frames of Chunk 214, corresponding to the scene where the flying squirrel is flying over the forest floor right before the spikes pop up.

I’ve used contextual shading here to show the quantization parameter for each frame. In VC-1, a 1.0 is the least compressed and 31 is the most compressed. In general, QP is a decent proxy for the quality of a given frame, with values below 8 generally being pretty good and values above 8 getting increasingly blocky.

Since this is a 2-second chunk at 24 frames a second, we have exactly 48 frames. And since it’s a Closed GOP, it starts with an I-frame (keyframe).

The Insane, HQ, and Fast settings offer pretty similar results, due to using the 2 B-Frame pattern and Lookahead to detect scene and other changes. However, QP generally goes down as encode time goes up. The Default encode starts out with much higher QP since the previous scene had used more, and so needed to catch up a bit. By the end of the GOP, its QPs are going down while the “2 B” QPs are going up for a similar reason.

Interesting in itself but it’s definitely a trees-over-forest view.

For 95% of the 10+ minute clip, all the frames look fine; as typical with a CBR encode. It’s in the hard parts where the differences show up.

For the below, I’m going to focus on chunks 209-221 (6:56 to 7:22), which includes the very challenging section where the squirrel is flying over all the spikes jumping up.

In the below, we’re looking at a few metrics here:

• Mean QP per chunk:  The average QP of all frames in the chunk, as a proxy over overall quality.
• Max QP per chunk:  The highest QP in the chunk. Since it’s the ugliest parts of the video that stand out, that’s also an important flag for quality issues.
• Chunk size:  How many bytes in the chunk. Since Smooth Streaming is all about delivering video chunks, chunk size is the primary measure of data rate.

Here’s the data:

You can see there’s quite a bit of variability in chunk size; almost 3:1. That’s fine; we’d expect to see that in content that alternates between easy and hard shots. The swing between Mean and Max QP is worst for the Default settings; Lookhead is handy to help the codec know what’s coming down the pike. For Chunk 217, we see Default has a lower Mean QP but a higher Max QP compared to the others. In general, the Mean and Max QP is highest in the Default, as we can see in the below graphs:

Overall, we see that Default is the weakest, and Insane the strongest. But the gap from Default to Fast is the biggest change; in most chunks, Insane and HQ offer the same Max QP, and HQ has a better mean QP in a few cases. However, there are some chunks where Default does better than any of the other modes; since it’s a 1-pass CBR, one mode may have used more of its buffer by a given point than another, leaving fewer bits for the next part.

For reference, here’s that QP 31 frame in our four versions (click on the thumbnails to open at full resolution).

Default:

Fast:

HQ:

Insane:

So, what went wrong with that particular frame? It was a rate control failure of some sort. We can see from the below that the surrounding frames got more bits and lower QP in Default; it would have been better with a more consistent compression level. It’s also interesting to note the big improvement from Fast to HQ for these sequence of frames; it’s when the content is the most challenging that the slower, higher quality encoding modes make the most significant difference.

When you watch the video at full speed, the QP 31 frame isn’t really that noticeable as it’s right at a scene change.

Given the structure of Smooth Streaming the optimum way to allocate bits is quite a bit different than “classic” web encoding with the VC-1 Encoder SDK used in Expression Encoder was tuned for. Any compression engineer reading this has probably already thought of a dozen ways to improve encoders for chunked video. More about that later.

Lastly, here’s how long each version took to encode.

• Default: 59 minutes
• Fast: 59 minutes (1.0x Default)
• HQ: 158 minutes (2.68x Default)
• Insane: 268 minutes (5.4x Default)

As you can see, you rapidly get to the point of diminishing returns with slower encoding modes; the biggest improvement was from Default to Fast, which had no impact on encoding time at all.