Now that same HD streaming is available in Media Center too. Ian Dixon reports that Netflix members running Windows 7 will have access to over 1800 movies in HD in Windows Media Center. In addition to streaming, from within the WMC Netflix app, you can also search your Netflix library, manage your Instant and DVD queues, filter searches and more.

For more information, check out NBCSports.com.

I gave a quick response in the comments that I wanted to expand into a proposed definition.

Resolution and frame rate

The infamous Table 3 of the ATSC specification lists 18 different resolution and frame rate combinations for digital broadcasting, But they drew a pretty clear line on what’s HD and what’s not: 1280x720 is HD, and anything less isn’t.

A more pedantic definition could be that HD should be at least 1920x1080p24 or 1280x720p60. And we’ll get there on the web before too long. But for now, I want the full 720p experience as a minimum bar. I can see two flavors of that which we can define as the ragged low end of web HD; anything below these are something else.

1280x528p24

For film source content, the widest aspect ratio in common use in 2.4:1. With 1280 wide, if we crop to the active image area and then round down to the next divisible-by-16 value for optimum compression, we’re left with 1280x528. For 1.85:1 movies, the equivalent is 1280x688

960x720p30

Lots of production codecs for HD aren’t square pixel. HDCAM is 1440x1080, and DVCPROHD is 1280x1080 or 960x720, even though both formats are always 16:9. So, I’m also inclined to allow 960x720 anamorphic for HD, but only in 30p. DVCPROHD’s 24p mode is 1280x1080p24.

Thusly…

If we calculate the pixels per second of the above, we get:

• 1280x528x24=16,220,160 pixels/sec
• 960x720x30=20,736,000 pixels/sec

So the anamorphic is still more pixels/sec than square pixel 2.4:1. We could arguably define “Web HD” as “at least 16M pixels/second” as well.

Quality

Hassan Wharton-Ali brought up another good point on the thread - HD should actually be HD quality. It can’t be a lousy, over-quantized encode using a suboptimally high resolution just so it can be called HD.

A good test is the video should look worse (due to less detail), not better (due to less artifacts), if encoded at a lower resolution at the same data rate. If reducing your frame size makes the video look better when scaled to the same size, then the frame size is too high!

Samples!

I know these are going to be too wide for my default blogging interface, but I need to stick an iframe somewhere in order to lock down the frame size exactly, due to how Silverlight Streaming works. Autoplay is turned off so your browser doesn’t automatically start two HD clips at once.

NOTE, yes I know in fact they still are doing Autoplay for some reason. I'm trying to fix that right now. Just make sure to pause the one you aren't watching. At least I got the size locked down (although still with too much padding...).

These are WMV files, but encoded with EEv2 SP1 using Smooth Streaming settings (so 1-pass CBR, 2 sec Closed GOP, and all that jazz). These are pretty basic encodes; I didn’t do anything tricky, as we WANT to see some artifacts as we’re defining these as the low bound of HD and below the low bound of HD. I do three versions of each clip

• True HD at 3 Mbps (64 Kbps audio, 2937 Kbps video)
• “Fake” HD at 1 Mbps (32 Kbps audio, 959 Kbps video)
• Good SD (640x360 for 16:9, 640x264 for 2.4:1) at 1 Mbps (48 Kbps audio, 943 Kbps video)

The SD is in there as an anchor to show that a lower resolution can actually look better at lower bitrates. Clearly the SD looks better most of the time, so 1 Mbps doesn’t count as HD. And those 16 Kbps of audio make a real difference with WMA 10 Pro, taking us from 32 to 44.1 KHz.

If you mouse the player, it’ll pop up controls (but without scaling the video – one of the nice features of the Black Glass template). To pick a particular clip, click on the icon that looks like poker chips (although upon reflection I note they’re actually film reels).

The Island: 1280x528p24

I haven’t used this clip in ages, but it’s always great to trot out as an edge case of hard encoding. It’s a Michael Bay joint, full of whip-pans, super-fast editing, frenetic motion, and film grain. Low bitrates can look fine for 80% of the frames in a clip, but fall apart for 20%. This clip has some good sequences where there’s multiple edits a second, so you get lots of chances to see the bandwidth stress. If only we had an encoder that could dynamically adjust frame size based on content complexity…

Lady Washington: 960x720p30

Just a short version of it this time, mainly because I’m tired of slow DSL uploads. The HD sizes are anamorphic, the SD is square pixel.

It’s full of awesomeness, but I wanted to excerpt the section describing their 1st gen, 2nd gen, and HD encoding settings and workflow.

First Generation Encoding

Our first set of encodes are based on WMV3 and WMA in ASF with WMDRM10 (Janus). We chose these standards because the Janus components have been widely adopted by our CE partners such as Roku, LG Electronics, Samsung, TiVo, and of course Microsoft Xbox.
We encode most content at 500, 1000, 1600, and 2200kbps VBR, but some titles whose source quality merits it have also been encoded at 3400kbps. The highest bitrate encodes are fit into 720x480 non-square pixels (the usual 1.2 PAR for widescreen content, 0.9 PAR for 4:3), but optimum encoding at lower bitrates is achieved with fewer pixels. Encoded films are normally at 24fps to match the source, while shot-to-video and mixed material is de-interlaced to 30fps (or 25fps for PAL content).

Netflix has been using anamorphic video all along, which I think is an underused feature of Windows Media and other formats. When you’re limited to 720x480 pixels, you want to encode all the pixels you’ve got, without having to synthesize any extra.

Second Generation Encoding

The new Silverlight player (that some users are helping us test as I write) uses VC1 Advanced Profile encoding with PlayReady DRM. A key property is that each GOP header includes frame size and resolution, which allows us to assemble a stream on the fly from different bitrate encodes as your broadband bandwidth fluctuates. (Another key feature is more coverage, including Intel Macs and Firefox users.) We expect to switch completely to the new player later this year.
The VC1 encoders are more efficient than the WMV3 encoders, so we are currently encoding VC1AP at slightly lower birates: 375, 500, 1000, and 1500kbps, all square pixel. At some point we are likely to add a couple more resolutions of non-square pixel encodes capturing the original pixel-aspect-ratio of the source.
We are also re-wrapping the VC1AP encodes in WMDRM10 for CE devices, which will gradually switch to the more efficient encodes in future firmware upgrades.

This is a great example of the improved efficiency of the VC-1 Encoder SDK and tools based on it. Not all Windows Media/VC-1 encoding is equal; the latest tools can offer a very meaningful reduction in bitrate required for a give quality level, improving user experiences and the cost of delivering the content.

The new encodes are backwards compatible to older hardware and software encoders, so even older devices can take advantage of the improvements.

High Definition Encodes

Today we have rights to deliver about 400 streams in HD (720p). More titles will be added over time. We experimented with first-generation WMV3 encodes at 4000kbps and 5500kbps, but settled on second-generation HD encodes with VC1AP at 2600kbps and 3800kbps, which extends their accessibility down to lower home broadband connections. As with SD, encodes of film material are at 24fps, and encodes of shot-to-video material are at 30fps (or 25fps for PAL), rather than the 60fps that would come from a Blu-ray disc - we judged the 60fps content as too expensive of bandwidth for now. In general, these encodes are definitively better than SD, but won't challenge well-executed Blu-ray encodes - that would require a bitrate out of reach for most domestic broadband today. We believe Moore's law will drive home broadband higher and higher enabling full 1080p60 encodes in a few years.

There’s codecs for you – a good 720p experience at 3800 Kbps, which is probably a little below the average 480p bitrate used for MPEG-2 on the DVDs that fly around the nation in those Netflix envelopes.

I’m not that worried about 1080p60 encoding myself. The vast majority of fictional content is shot 24p, including 99% at least of Blu-ray discs. While Blu-ray players may output 1080p60, the encodes, and hence bitrate requirements, are still 1080p24. I find that typical film content in VC-1 wants around 6-8 Mbps for a 1080p24 experience enough better than 720p to be worth the trouble.

I just got my Xbox set up to handle the Netflix streams – now I’ve got to go check out some of those 400 HD titles! Hmm. Pan’s Labyrinth and Heroes.

Anyway, it’s great stuff. I love it when partners can share this kind of detail about what they’re doing.

Compression, although an obsession with me since I was 19, didn't appear to be a career option until many years after that. My years at Hampshire College were spent essentially majoring in neuropsychology, minoring in computer science, and spending my evening and weekends helping out my film student buddies. It all seemed hopelessly random to my parents and advisors, but turned out to be the perfect background for what I do now (after all, what's compression but extremely applied neuropsychology?).

After college and a couple of science internships under my belt I decided I didn't want to spend my life writing grant proposals or doing lab work so I started a video production company with my friends, including my recent interviewer Halstead York. The plan was to use emerging technology to be able to produce and post independent films from our own scripts. We thought we had a financing deal lined up back in 1994, and purchase a NLE: (a PowerMac 8100/80 with a Radius VideoVision card, and 4GB SledgeHammer RAID) was purchased for doing video editing. The idea was we could rent it out before and after post in order to cover some of the costs. Then there were two big problems:

1. The infamous defective BART chip in those early PowerMacs meant it couldn't keep sync for more than a few minutes.
2. Our financing fell through.

So, there we were, with a script, no money, a bunch of debt, and a NLE that couldn't edit video. However, we found a nice market using shorter clips with looser sync requirements: CD-ROM video! And so we were launched in the heady early days of multimedia. Journeyman Digital was a full service production company for digital media, and we did all the screenwriting, production, and post that we dreamed of, but not for our own projects. But we kept writing screenplays on the side. We got as far as a few meetings with Sony Pictures on one, but like nearly all screenplays, nothing really happened in the end. And while I liked doing the work, when it came down the the fundamental gut check of moving to LA and rolling the dice, I didn't NEED to do it. Instead I got married and soon enough had three little kids, and rather ran out of time for side projects.

Halstead is only recently married and currently kidless, and had time. So he and many members of the old gang dusted off one of our old screenplays, Temporary Insanity and darn if it they didn't actually shoot the whole thing in HD! Halstead just finished up the trailer. Quite an experience seeing jokes I wrote a decade ago there on the screen. And it's amazing to see how it's finally possible to make movies on a hobbyists budget, even with high-end techniques. Check out this post on color correction in the home office.

I didn't have time to work on the production itself (I was busy having that third child get born and joining Microsoft), but I certainly wasn't going to let anyone else compress the trailers (now available for download)!

The project

And so, after all that ramble, we're back to talking about hands-on compression.

Halstead had a pretty typical 2x2 matrix for encoding: two formats at two data rates each:

Formats

• MPEG-4 compatible with QuickTime/AppleTV/iPod
• Windows Media compatible with Windows Media Player/Flip4Mac/Xbox/Zune/Silverlight

Data rates

• 3 Mbps for a 720p30 HD version compatible with Xbox360/AppleTV
• 300 Kbps for a low data rate download, which would also be portable media player compatible (iPod for .mp4, Zune for .wmv)

Workflow

The source was provided as a 730p30 .AVI file using the CineForm Aspect HD codec. It was video-only - audio was provided in a separate .wav file.

HD WMV encoding was easy - I was able to use the source as is. And the current WMCmd.vbs supports specifying a separate .wav file as source for the audio track.

HD .MOV was harder. I wanted to use QuickTime's H.264 encoder to output, since it uses a complexity-constrained mode that is well tuned for computer playback via QuickTime, on both Intel and PPC (and there's a lot of G4 PowerBooks out there among Indie film fans). While it won't offer the same compression efficiency as a highly-tuned H.264 encoder from another encoder, it'll also playback well on more machines.

However, QuickTime, even QuickTime for Windows, can't read AVI files using the standard DirectShow API! Now that we've added support for the QuickTime API in Expression Media Encoder, it's only fair for Apple to support DirectShow now . So, I used Rhozet Carbon to encode my .avi and .wav source files into a single Photo-JPEG compressed .MOV file that QuickTime could then read (believe it or not, there's no lossless Y'CbCr 4:2:0 encoder in QuickTime for Windows). I wound up doing that compression on my G5, so I could do it in parallel with the WMV encoding on my Windows box.

For the mobile versions, I used VirtualDub to make me a nice 320x180 version of the .AVI and Carbon again to make a 320x180 JPEG .mov.

As an alternative (and what I would have done if this was going to be a high-volume process and not just a one-off) would be to using Carbon to encode all four outputs from the single source. Also, using the "multipass" mode with Carbon and other tools other than QuickTime Player Pro itself results in very, very slow rendering time, since it reruns preprocessing for the entire clip for each pass, although only a small part of the file might be adjusted per pass. So in a high-volume workflow, probably only the 1-pass mode would have been used.

Windows Media Settings

WMV HD @ 3 Mbps:

cscript "C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs" -input "G:\Temp Insanity\Trailer 1 timed v5 720.avi" -output "Trailer 1 720p 3M 192.wmv" -a_input "G:\Temp Insanity\Trailer 1.wav" -a_codec WMASTD -a_mode 4 -a_setting 128_48_2 -v_codec WVC1 -v_mode 4 -v_keydist 5 -v_bitrate 2870000 -v_peakbitrate 6000000 -v_peakbuffer 4000 -v_performance 80 -v_bframedist 1 -v_dquantoption 2 -v_loopfilter 1 -v_mmatch 0 -v_mslevel 4 -v_msrange 0 -v_percopt 2

Pretty standard stuff, with the same basic settings as my previous encodes. A few items of note:

1. Not excessive vertical motion and HD, so I didn't bother constraining the number of threads.
2. Since the source was just stereo, I used WMA instead of WMA Pro, in order to preserve Silverlight 1.0 compatibility.
3. Note the use of the -a_input flag to specify a different audio source.

WMV mobile @ 300 Kbps:

cscript "C:\Program Files\Windows Media Components\Encoder\WMCmd.vbs" -input "Trailer 1 timed v5 320x180.avi" -output "Trailer 1 280 Zune.wmv" -v_codec WMV9 -v_mode 4 -v_keydist 10 -v_bitrate 235000 -v_peakbitrate 600000 -v_peakbuffer 4000 -v_performance 80 -v_bframedist 1 -v_loopfilter 1 -v_overlap 1 -v_mmatch 0 -v_mslevel 2 -v_msrange 0 -v_percopt 2 -v_numthreads 1 -a_codec WMASTD -a_mode 4 -a_setting 48_44_2 -a_peakbitrate 160000

Pretty much identical to the Zune encoding settings I posted last week, except with lower data rates to hit the 300 Kbps total.

1. The audio was pretty simple, so 48 Kbps was enough when using VBR mode (again VBR audio is a very underused and very useful feature for downloadable files).
2. the data rate was so low, I went to the max and used -mslevel 2 (full floating point chroma search) and -v_numthreads 1 (single-thread encode). Even with those, this encoded much quicker than the HD version, since the frame size was so much smaller.
3. Main Profile is required by Zune, and thus I can't use DQuant.

QuickTime Settings

QuickTime's advanced settings aren't available via command-line, so I'll include screen shots of my MPEG-4 settings.

I matched the WMV settings as closely as appropriate.

MPEG-4 Main Profile @ 3 Mbps

1. The "Current" mode passes through the source frame size and frame rate (Note it would have said 1280x720 (Current) above - I had a different source loaded when I took the screen shot).
2. "Optimize for Download" is the equivalent of our 2-pass VBR modes. However it lacks the ability to specify a peak buffer rate or duration.
3. QuickTime specifies keyframe rate in terms of total frames between keyframes, not total seconds.
4. The "Better" mode for audio encoding quality is optimal for 16-bit sources. The "Best" mode only improves >16-bit sources
5. The Multi-pass mode improves quality, but can make encoding time very unpredictable. The WMV versions encoded quite a bit faster on a similar era machine (Dual 3.4 GHz "NetBurst" Xeon versus dual 2.0 GHz G5). My main compression box, a quad AMD, was busy doing some other work.
6. QuickTime lacks a true 2-pass VBR audio mode. For MPEG-4 exports, I only get 1-pass CBR. With a QuickTime export, I could have gotten a 1-pass VBR encode, but only in a MP3 style "range" encode, where the final file size could vary substantially. For soundtracks in downloadable files, this makes WMA a more efficient codec.
7. Main Profile is compatible with AppleTV, and uses B-frames. The "Extended" profile is theoretically for streaming, but it's been grayed out in QuickTime since H.264 support launched in QuickTime 7.0, and I've never seen a H.264 Extended Profile stream in the wild.

The mobile encode was the same, except with the lower video and audio data rate, and its use of the Baseline profie, required for iPod compatibility.

Differences

So, how did the two encodes come out?

For the most part, they both looked and sounded good (or at least accurate - the audio mix will be improved in a later version). The biggest difference was in flatter areas, especially with shadows. That's where the VC-1 Differential Quantization and Perceptual Optimization come in, plus the ability to use different block sizes(4x4, 4x8, 8x4, and 8x8), to better compress the edges and interiors of flat areas. The Baseline and Main Profiles of H.264 are limited to 4x4 blocks only, and H.264 doesn't have an equivalent mechanism to DQuant to compress flat areas of the image less.

Again, another H.264 encoder could have done a better job here, although at the cost of higher decode requirements, by using features like CABAC and multiple reference frames. High Profile, and hence 8x8 blocks, are not compatible with QuickTime's H.264 decoder, nor those in the AppleTV or iPod. The iPod-required Simple Profile doesn't support B-frames or CABAC.

Here's some samples from the available clips that show different levels of banding. Sorry the luma levels don't quite match - it's surprisingly difficult to get exact level screen grabs out of the QuickTime and DirectShow pipelines. If anything, these minimize the banding seeing in the clips when looking at them in QuickTime on a Mac (2.2 to 1.8 gamma correction issue?).

H.264:

VC-1:

H.264:

VC-1:

Needless to say, this makes it a lot easier for third parties to build HD DVD players! And we’re announcing that Lite-On IT Corp. and Zhenjiang Jiangkui Group Co. Ltd./ED Digital plan to use this new hardware and software platform. Lite-On is one of the most most experienced systems integrators backing high-volume consumer electronics . Zhenjiang Jiangkui is one of the highest volume DVD player manufacturers in China.

Additionally, we’re working with Meridian Audio Ltd to build a high-end HD DVD player for the enthusiast market.

Shinco and Onkyo will also create HD DVD players.

Of course, we continue to work with Sonic Solutions, CyberLink, InterVideo, Ulead, and Nero on their software players. Lastly, our HDi technology continues to be used in the three second generation Toshiba players, as it was in the first generation.