Expression® Encoder 3 – Part 1 of 2 Screen Capture
What is it?
Expression Encoder 3 provides the power of industry leading encoding, in a simple approachable interface to make it easy to prepare video for use in a variety of ways, including Silverlight on the Web.

Download Site: http://www.microsoft.com/expression/try-it/

Example Problem(s) it solves:
Encoder has a screen capture application to record from your screen with webcam and microphone support to produce demo and training materials

Usage Notes: Expression Community Home: http://expression.microsoft.com

• AT&T
• CD Networks
• Internap
• Level 3
• Limelight

• Digital Rapids
• Envivio (and press release)
• Grab Networks (formerly Anystream)
• Inlet (and press release)
• Rhozet
• Telestream (and press release)
• VBrick
• Viewcast (and press release)
• Winnov

Which is a huge swath of the professional compression tools market.

DRM service provider support
And we've had a bunch of support for Silverlight DRM powered by PlayReady from DRM service providers. Since Silverlight encryption is applied during content creation, not during content distribution, using DRM has no real impact on the server side; access to a DRM license server to provide licenses to the Silverlight client is the only big difference from a service perspective.

• BuyDRM (press release)
• Entriq
• ExtendMedia 
• Ipercast

• iStreamPlanet

I haven't had a chance to drop by everyone's booth yet, but will be highlighting some of their demos and cool announcements throughout the week when I find out which of the many projects we've been collaborating on are public now.

Silverlight 3: we're 1080p24
So, we had that whole HD on the web discussion a few weeks ago which spun out into many threads on different forums. A few die-hards said that only 720p60 and 1080p24 should count as full HD, based on the original ATSC definition.

It'll keep on being discussed, but that won't keep anyone from calling Silverlight HD, because we've now got 1080p24 working in Silverlight 3. This represents a huge amount of media playback tuning and testing by the Silverlight team, and it's really paid off. Much of the improvements have been since the public Silverlight 3 beta, so those not at NAB will have to take my word for it for the moment, but we're showing off:

• 1920x1080p24 VC-1 at 6 Mbps
• On a Core 2 Duo 2.4 GHz system
• With smooth playback at any size using GPU scaling

And it is glorious.

On the same system we're also showing off H.264 720p24 2.5 Mbps with similarly smooth playback.

While those specs are obviously beyond what many home PCs can do due to screen size or network speed, with Smooth Streaming we can offer that as the highest-end experience while offering further bitrate bands as low as required.

And of course, optimization that enables 1080p on the high end makes 480p work on lower-end machines than Silverlight 2 could support, due to both media pipeline improvemetnts and offloading scaling and compositing to the GPU. The gains on single-core machines are particularly notable; we've got a quite nice Smooth Streaming experience even on NetBooks now.

The public release of Silverlight 3 will be later this year.

New Smooth-Streaming VC-1 implementation
We're working with the encoder tool vendors to integrate a new VC-1 implementation that's specifically tuned for Smooth Streaming. You may recall Expression Encoder's  Smooth Streaming mode uses the VC-1 Encoder SDK in 1-pass CBR mode with a fixed GOP size. While that certainly can produce good video that's Smooth Streaming compliant, in the end that's really the kind of settings used for live broadcasting. With on-demand content, we can do an analysis pass to figure out a variety of better ways to optimize the bitrate. In particular, we can dramatically reduce the incidence of frames compressed to the point where visible blocky artifacts appear.

What it does are awesome in so many ways that it makes my compression nerd soul twinkle with delight. But those details will have to wait for a long and Excel-chart laden blog post of its own.

In the interim, let me offer you a sample encoded in a not-quite-final version of what we're doing here. This is our favorite Big Buck Bunny content, encoded to these specs

• 1080p24 4 Mbps video with a 5 second buffer
• Smooth Streaming compatible VC-1 and WMA 10 Pro
• In a WMV wrapper, so you can play it in WMP

This is what the top end Smooth Streaming bitrate can look like once Silverlight 3 is released later this year. And as mentioned above, we're now doing 6 Mbps on a Core 2 Duo, we've got some further headroom for more challenging content.

Here's the file (hosted by Silverlight Streaming):
http://silverlight.services.live.com/31260/Big%20Buck%20Bunny%201080p24%204%20Mbps%20Smooth%20Streaming/video.wmv

Remember, Windows Media Player can Save As progressive download content without DRM if you want to make a local copy.

If you're here at NAB, Inlet is showing off the new VC-1 implementation as implemented in their Armada product, with a bunch of other output samples. Check it out.

I gave a quick response in the comments that I wanted to expand into a proposed definition.

Resolution and frame rate

The infamous Table 3 of the ATSC specification lists 18 different resolution and frame rate combinations for digital broadcasting, But they drew a pretty clear line on what’s HD and what’s not: 1280x720 is HD, and anything less isn’t.

A more pedantic definition could be that HD should be at least 1920x1080p24 or 1280x720p60. And we’ll get there on the web before too long. But for now, I want the full 720p experience as a minimum bar. I can see two flavors of that which we can define as the ragged low end of web HD; anything below these are something else.

1280x528p24

For film source content, the widest aspect ratio in common use in 2.4:1. With 1280 wide, if we crop to the active image area and then round down to the next divisible-by-16 value for optimum compression, we’re left with 1280x528. For 1.85:1 movies, the equivalent is 1280x688

960x720p30

Lots of production codecs for HD aren’t square pixel. HDCAM is 1440x1080, and DVCPROHD is 1280x1080 or 960x720, even though both formats are always 16:9. So, I’m also inclined to allow 960x720 anamorphic for HD, but only in 30p. DVCPROHD’s 24p mode is 1280x1080p24.

Thusly…

If we calculate the pixels per second of the above, we get:

• 1280x528x24=16,220,160 pixels/sec
• 960x720x30=20,736,000 pixels/sec

So the anamorphic is still more pixels/sec than square pixel 2.4:1. We could arguably define “Web HD” as “at least 16M pixels/second” as well.

Quality

Hassan Wharton-Ali brought up another good point on the thread - HD should actually be HD quality. It can’t be a lousy, over-quantized encode using a suboptimally high resolution just so it can be called HD.

A good test is the video should look worse (due to less detail), not better (due to less artifacts), if encoded at a lower resolution at the same data rate. If reducing your frame size makes the video look better when scaled to the same size, then the frame size is too high!

Samples!

I know these are going to be too wide for my default blogging interface, but I need to stick an iframe somewhere in order to lock down the frame size exactly, due to how Silverlight Streaming works. Autoplay is turned off so your browser doesn’t automatically start two HD clips at once.

NOTE, yes I know in fact they still are doing Autoplay for some reason. I'm trying to fix that right now. Just make sure to pause the one you aren't watching. At least I got the size locked down (although still with too much padding...).

These are WMV files, but encoded with EEv2 SP1 using Smooth Streaming settings (so 1-pass CBR, 2 sec Closed GOP, and all that jazz). These are pretty basic encodes; I didn’t do anything tricky, as we WANT to see some artifacts as we’re defining these as the low bound of HD and below the low bound of HD. I do three versions of each clip

• True HD at 3 Mbps (64 Kbps audio, 2937 Kbps video)
• “Fake” HD at 1 Mbps (32 Kbps audio, 959 Kbps video)
• Good SD (640x360 for 16:9, 640x264 for 2.4:1) at 1 Mbps (48 Kbps audio, 943 Kbps video)

The SD is in there as an anchor to show that a lower resolution can actually look better at lower bitrates. Clearly the SD looks better most of the time, so 1 Mbps doesn’t count as HD. And those 16 Kbps of audio make a real difference with WMA 10 Pro, taking us from 32 to 44.1 KHz.

If you mouse the player, it’ll pop up controls (but without scaling the video – one of the nice features of the Black Glass template). To pick a particular clip, click on the icon that looks like poker chips (although upon reflection I note they’re actually film reels).

The Island: 1280x528p24

I haven’t used this clip in ages, but it’s always great to trot out as an edge case of hard encoding. It’s a Michael Bay joint, full of whip-pans, super-fast editing, frenetic motion, and film grain. Low bitrates can look fine for 80% of the frames in a clip, but fall apart for 20%. This clip has some good sequences where there’s multiple edits a second, so you get lots of chances to see the bandwidth stress. If only we had an encoder that could dynamically adjust frame size based on content complexity…

Lady Washington: 960x720p30

Just a short version of it this time, mainly because I’m tired of slow DSL uploads. The HD sizes are anamorphic, the SD is square pixel.

The IIS team has just released the first public beta of the Smooth Streaming module for IIS7. This means anyone with Windows Server 2008 can start experimenting with this next generation media delivery technology for Silverlight. We’re aiming for a final release in Q2 this year, which will be licensed for full commercial use. There’s lots of detailed info in the links before, but let me offer my own high-level take on what Smooth Streaming is about and why it matters.

Smooth Streaming adapts streaming to the web instead of trying to adapt the web to streaming

Alex Zambelli explored this concept in the links below, but I love it so much I’m going to reiterate. Smooth Streaming leverages the massive scale of the standards-based http web delivery technologies, instead of trying to define different protocols or requiring a lot of bidirectional communications. The media itself is sent as a series of small files as http requests, so it gets automatically cached by the proxy servers in CDNs, ISPs, and corporate firewalls. And of course can pass through firewalls easily.

Smooth Streaming streams smoothly

The core user experience we’re shooting for is quick starting, fast scrubbing, uninterrupted playback of content as good as the local machine can play at the moment. We switch between bitrates and resolutions basic on available CPU speed, network speed, and window size without any “buffering” messages as long as there’s sufficient bandwidth to deliver the lowest bitrate.

Smooth Streaming is standards based

Smooth Streaming uses the http protocol and the MPEG-4 file format (in its “fragmented” flavor). We’re initially using VC-1 (SMPTE standard) and WMA 10 Pro audio, and will support H.264 and AAC-LC support once Silverlight 3 ships with those codecs. The manifests use XML and SMIL.

Smooth Streaming can be authored today

Expression Encoder 2 (with Service Pack 1 installed) includes full support for authoring on-demand Smooth Streaming content, including all media files and manifests. The Silverlight 2 templates included with SP1, and the underlying Base Player, support Smooth Streaming. Other compression tools will be adding full Smooth Streaming support in future versions.

All the Links

We’ve gone blog-crazy here at Microsoft. Which means that there’s lots of great information coming out of us these days, so it can be kind of hard to find it all. Here’s my list of good Smooth Streaming references and tutorials.

Official pages

• IIS Smooth Streaming
• IIS Media

The Demo

• SmoothHD.com (our joint demo site with Akamai)

Downloads

Yep, it’s a free beta download for any version of Windows Server 2008. There are free 60-day trial versions of 2008, including a .vhd download.

• Smooth Streaming Beta for 32-bit Windows Server 2008
• Smooth Streaming Beta for 64-bit Windows Server 2008
• Web Platform Installer (single installer for IIS, SQL Server Express, .NET Framework and Visual Web Developer components)
• Trial versions of Windows Server 2008
• Expression Encoder 2 Service Pack 1 (the full Expression Encoder 2 with SP1 installed is needed to author Smooth Streaming content)

Learn IIS7 articles

• Smooth Streaming for IIS 7.0 - Getting Started
• Smooth Streaming for IIS 7.0 - Exploring Bit Rate Changes
• Smooth Streaming for IIS 7.0 - Managing Your Presentations

My Blog

• Expression Encoder 2 Service Pack 1 – Intro and Multibitrate Encoding (my general overview of Smooth Streaming)
• Encoding Big Buck Bunny for SmoothHD.com (highy nerdy hands-on example… with tables and charts!)

Alex Zambelli

Alex has been doing a great series on multibitrate streaming, adaptive streaming, and Smooth Streaming architecture from a content authoring perspective. They’re great to go through in order

1. A Brief History of Multi-Bitrate Streaming
2. The Birth of Smooth Streaming
3. Smooth Streaming Architecture

John Bocharov

John is an IIS Program Manager who has been intimately involved with Smooth Streaming since its inception.

• Beta Release of Smooth Streaming Now Available
• Smooth Streaming Questions? We’ve Got Answers!
• I’ve seen the future and the future is… Smooth!

I figured this was as good a time as go through a hands-on project for Smooth Streaming.

The Source

We’ve had a few years of good fun with the Blender Foundation’s Elephant’s Dream clip (as I used here, here, and here). They have a new Creative Commons licensed title up called Big Buck Bunny, which is a lot brighter, and with a lot of nice chewy grass for high frequency detail. Elephant’s Dream is a great test clip for handling gradients and dark detail (It’s all about differential quantization), but Big Buck Bunny is arguably a better test for general compression quality, and also has some nice sky gradients as well.

I downloaded the stereo audio as a FLAC file, and converted that to a 44.1 KHz 16-bit WAV file with Audacity. I then downloaded source as a series of PNG files, and then used After Effects to combine the PNG sequence and the WAV that into a 1920x1080 Lagarith RGB AVI file. Since PNG and Lagarith are both lossless, and I didn’t do any scaling or color space correction, I didn’t do any preprocessing, just assembling the content into a file Expression Encoder 2 can consume. I’m a big fan of the quality of EEv2’s Super Sampling scaling mode, so prefer to use that instead of other tools where I have less control over scaling quality.

Setting up the Encode

Since we had other encodes up at SmoothHD.com, I wanted to follow the same data rate/frames size/GOP length parameters we’d already used. For the other encodes in the series. Specifically, the below, which give us seven bands from 2500 Kbps to 364 Kbps:

Tweaking

Smooth Streaming as a feature came in very hot for EEv2 SP1, and there wasn’t a lot of time to tune the presets.

There are a couple of safe tweaks I always recommend as previously mentioned:

• Adaptive Dead Zone=Conservative
• Overlap=On
• Scene Change Detection=On
• Search Range=Adaptive

Those help Smooth Streaming (and web-rate VC-1 in general) quality with minimal impact on encoding time.

For quality-over-speed projects, I also like to use

• Complexity=4
• Chroma Search=Full True Chroma
• Match Method=Adaptive

All together, that’ll increase encode time about 2-3x for a nice but not overwhelming quality difference.

And because this is animation content, I’ll also use

• B-Frame Number=2

In general, the less noise in the content and the closer the matches between frames, the more likely 2 B-Frames will be more efficient than 1. I’ve gone up to 4 B-frames for Camtasia screen recordings.

Charts!

One cool thing about Smooth Streaming encoding is that the independent chunks make analysis and graphing of the content a lot easier. We can analyze each chunk as an independent unit, and compare quality between bitrates and versions quite directly. Do note that there is some allowed variability in chunk size; we target for a 5 second buffer, so that any three chunks in a row should average around the same amount, but if a particular chunk is harder to encode than its neighbors, it can have a somewhat higher bitrate.

Here’s the .XLSX if you want to check out the source data:

I’ve encoded four versions of the content

• Default: the stock EEv2 SP1 Adaptive Streaming settings
• Fast: Adding the “safe tweaks” above
• HQ: Adding the quality-over-speed settings
• Insane: Taking HQ and adding "Insane” modes that add a lot more in encoding time than quality
  • Single-thread encoding
  • Complexity 5
  • Hadamard Motion Match

Insane is there to show what happens when you burn as much CPU as possible.

To show the impact of these different encoding settings on the video, I used an in-house command-line tool to dump a log file of the size, type, etcetera of each frame, and then did made the Excel spreadsheet linked above to see what interesting data I could find.

First off, let’s take a look at just a bunch of frames. Specifically, the frames of Chunk 214, corresponding to the scene where the flying squirrel is flying over the forest floor right before the spikes pop up.

I’ve used contextual shading here to show the quantization parameter for each frame. In VC-1, a 1.0 is the least compressed and 31 is the most compressed. In general, QP is a decent proxy for the quality of a given frame, with values below 8 generally being pretty good and values above 8 getting increasingly blocky.

Since this is a 2-second chunk at 24 frames a second, we have exactly 48 frames. And since it’s a Closed GOP, it starts with an I-frame (keyframe).

The Insane, HQ, and Fast settings offer pretty similar results, due to using the 2 B-Frame pattern and Lookahead to detect scene and other changes. However, QP generally goes down as encode time goes up. The Default encode starts out with much higher QP since the previous scene had used more, and so needed to catch up a bit. By the end of the GOP, its QPs are going down while the “2 B” QPs are going up for a similar reason.

Interesting in itself but it’s definitely a trees-over-forest view.

For 95% of the 10+ minute clip, all the frames look fine; as typical with a CBR encode. It’s in the hard parts where the differences show up.

For the below, I’m going to focus on chunks 209-221 (6:56 to 7:22), which includes the very challenging section where the squirrel is flying over all the spikes jumping up.

In the below, we’re looking at a few metrics here:

• Mean QP per chunk:  The average QP of all frames in the chunk, as a proxy over overall quality.
• Max QP per chunk:  The highest QP in the chunk. Since it’s the ugliest parts of the video that stand out, that’s also an important flag for quality issues.
• Chunk size:  How many bytes in the chunk. Since Smooth Streaming is all about delivering video chunks, chunk size is the primary measure of data rate.

Here’s the data:

You can see there’s quite a bit of variability in chunk size; almost 3:1. That’s fine; we’d expect to see that in content that alternates between easy and hard shots. The swing between Mean and Max QP is worst for the Default settings; Lookhead is handy to help the codec know what’s coming down the pike. For Chunk 217, we see Default has a lower Mean QP but a higher Max QP compared to the others. In general, the Mean and Max QP is highest in the Default, as we can see in the below graphs:

Overall, we see that Default is the weakest, and Insane the strongest. But the gap from Default to Fast is the biggest change; in most chunks, Insane and HQ offer the same Max QP, and HQ has a better mean QP in a few cases. However, there are some chunks where Default does better than any of the other modes; since it’s a 1-pass CBR, one mode may have used more of its buffer by a given point than another, leaving fewer bits for the next part.

For reference, here’s that QP 31 frame in our four versions (click on the thumbnails to open at full resolution).

Default:

Fast:

HQ:

Insane:

So, what went wrong with that particular frame? It was a rate control failure of some sort. We can see from the below that the surrounding frames got more bits and lower QP in Default; it would have been better with a more consistent compression level. It’s also interesting to note the big improvement from Fast to HQ for these sequence of frames; it’s when the content is the most challenging that the slower, higher quality encoding modes make the most significant difference.

When you watch the video at full speed, the QP 31 frame isn’t really that noticeable as it’s right at a scene change.

Given the structure of Smooth Streaming the optimum way to allocate bits is quite a bit different than “classic” web encoding with the VC-1 Encoder SDK used in Expression Encoder was tuned for. Any compression engineer reading this has probably already thought of a dozen ways to improve encoders for chunked video. More about that later.

Lastly, here’s how long each version took to encode.

• Default: 59 minutes
• Fast: 59 minutes (1.0x Default)
• HQ: 158 minutes (2.68x Default)
• Insane: 268 minutes (5.4x Default)

As you can see, you rapidly get to the point of diminishing returns with slower encoding modes; the biggest improvement was from Default to Fast, which had no impact on encoding time at all.

Miguel de Icaza’s blog has the full details and well worth the read. A few key points:

• This is an implementation of Silverlight 1.0. Thus it has
  • XAML for drawing the user interface
  • Browser JavaScript for application logic (no managed code yet)
  • Media Features
• The codec and file format support is distributed by Microsoft as a binary module that plugs into Moonlight. Users will be prompted to install it the first time they try to play back media. This allows for Microsoft to cover patent license fees associated with the codecs.
• Since we based that module on Silverlight 2 , this includes the VC-1 decoder improvements from Silverlight  2, and WMA 10 Pro.
• Moonlight is available in both 32-bit and 64-bit versions.

Now the team is moving on to Moonlight 2.0 and Silverlight 2 compatibility, which is currently in pre-alpha. The big work item there is getting the CLR/DLR fully integrated. Once we get that, we’ll have Smooth Streaming working on Linux! As an open source project, they’re welcoming help.

And here’s my favorite Moonlight project I just heard about: Moonshine. It uses Moonlight to emulate the old Windows Media Player OCX embedding model, so that web pages targeting the old player can work in Firefox on Linux. It can also play back local WMV files. Given that VLC hasn’t be able to play back VC-1 content with B-frames, I’m very glad for this.

Screenshot of Moonlight (taken from their page. I still need to get the new SUSE installed; my computers have all been encoding full blast since December).

Is VC-1 the same as Windows Media Video 9?

Yes, the big picture: VC-1 is the SMPTE standardized version (SMPTE 421M) of Windows Media Video 9. The output of of a VC-1 encoder is bitstream compatible with a Windows Media Video 9 decoder that supports the profile, level, and features used.

Is VC-1 the same thing as WMV?

Note that the VC-1 spec just defines the video bitstream; it doesn’t cover the ASF (used in .wmv and .wma) file format or any WMA codec. VC-1 Advanced Profile can live in a variety of other file format wrappers, including MPEG-4 (standard or fragmented, the latter being used in Smooth Streaming) and MPEG-2 transport stream (as used on Blu-ray or IPTV). Only VC-1 Advanced Profile has this kind of format independence; VC-1 Simple and Main profiles are always found inside a WMV file.

Can I use Expression Encoder to make files that play in Windows Media Player 9?

Yes, absolutely. VC-1 Main Profile is identical to the old Windows Media Video 9. A Windows Media file you make using VC-1 Simple and Main will work perfectly in WMP 9 out of the box. A stock XP Service Pack 2 machine that hasn’t been updated in years would play it without issue.

Will users have to install a update if I encode with VC-1?

No, everyone with Windows Media Player 9 or higher can play VC-1 Main Profile already. Windows Media Player 11 can play Advanced Profile as well. Users or WMP 9 or 10 will be prompted to install a VC-1 Advanced Profile decoder if they try to play a WMV file using that.

Can I encode for Blu-ray from Windows Media Encoder?

So, Since Windows Media Video 9 Advanced Profile is the same as VC-1 Advanced Profile, and VC-1 AP is supported on Blu-ray, can you encode a Blu-ray compatible stream in Windows Media Encoder?

Nope, sorry. There are further constraints for Blu-ray VC-1 that need to be followed, and WME and the Windows Media Format SDK weren’t designed to support that level of constraint. The VC-1 Encoder SDK was designed with optical discs in mind, and can produce a Blu-ray compatible VC-1 elementary stream with products that expose it. Expression Encoder 2 doesn’t do elementary streams, but Rhozet’s Carbon Coder  3.12 and higher do.

How do Windows Media Video 9 and VC-1 Profiles relate?

Here’s the details.

Following up on the above, there’s a couple of points I’d like to make clear

• Windows Media Video 9 originally had three profiles: Simple, Main, and Complex. Simple and Main are the same as the VC-1 Simple and Main Profiles. Complex was deprecated, and evolved into Advanced Profile. As Complex isn’t part of the VC-1 spec, it will normally only work on decoders from Microsoft, or based on Microsoft’s porting kit implementations.
• There were two flavors of Windows Media Video 9 Advanced Profile. The first, with a “WMVA” 4CC, was introduced with WMP 10 and used as the basis for VC-1 but some changes were made in the final specification that kept it from being entirely compatible. An updated VC-1 compliant version of Windows Media Video 9 Advanced Profile with a “WVC1” 4CC was introduced with WMP 11, and that is what should be used.
• Anyone wanting to make WMV 9 Advanced Profile needs to either have Format SDK 11 installed (the easiest way is to install WMP 11) or use a VC-1 Encoder SDK based product like Expression Encoder 2.

This is where Simple/Main/Complex profiles are set in Windows Media Encoder:

If it’s the same codec, why are there different Windows Media Video 9 and VC-1 Encoders?

All that said, there is one last practical difference between WMV 9 and VC-1: Encoders that call it VC-1 are normally based on the newer, faster, higher quality, more flexible VC-1 Encoder SDK. This isn’t about a change in the codec standard, just the usual improvement of encoder implementations over time, like we saw with the Format SDK 9, 9,5, and 11 releases, and will be seeing again in future releases.

Here’s a comparison of the gains from the initial release (1st generation) to VC-1 Encoder SDK (4th generation).

Further Reference

• http://alexzambelli.com/blog/wmv9-vc1-faq/
• http://alexzambelli.com/blog/vc-1-sdk-faq/
• http://www.microsoft.com/windows/windowsmedia/forpros/codecs/video.aspx#WindowsMediaVideo9AdvancedProfile

As I mentioned on Friday, the official streaming feed of Barak Obama’s Presidential Inauguration Committee will be using Silverlight 2.

However, using managed code in the player leaves out Linux users of the Moonlight 1.0 beta, as well as PowerPC Mac users. We’ve heard a lot of requests to add support for those platforms, and so we teamed up with the Moonlight team at Novell and they’ve created a Silverlight 1.0 version of the player that works great in both Moonlight and PowerPC Macs. Miguel de Icaza has details from his side.

We kicked the project off this afternoon before the event, so pardon us if it’s a little rough. The only significant limitation is that the SL1 player can only do the 500 Kbps stream – lacking managed code, making a manifest XML parser in JavaScript didn’t seem like something we should start testing only nine hours before the event goes live.

But anyway, thanks to Joseph, Larry, Geoff, Rusty and especially Aaron at Novell for pulling this together so quickly and so well, and Mio at iStreamPlanet for making the last minute encoder tweaks to make it Silverlight 1.0 compatible. On the Microsoft side, Brian Goldfarb and Eric Schmidt did a great job making the call and making it happen. Just go to www.pic2009.org in the morning and the updated player should be available around 6 am EST (11 am GMT).

That something like this could come together on such short notice is a testament to the chops and passion of the Moonlight team and the great platform we’re both implementing. I’m glad we got this chance to demonstrate how serious we are about this collaboration (although I wouldn’t have minded starting a day earlier…).

Update: The Silverlight 1.0 compatible player is low live at:

http://www.pic2009.org/page/content/linuxplayer

Note that this player is only for Silverlight 1.0 and doesn't work with Internet Explorer. The Silverlight 2.0 player for Windows or Intel Mac is at:

http://www.pic2009.org/page/content/live

(And yes, that would have been easy to fix, but we're rather out of time!)

Tech Specs

And as I promised earlier, I got the tech details on how the streams are encoded. Note there is manual stream selection in the lower right corner;

Hardware

There are quite a few encoders to handle the different streams, data rates, and to provide failover backups. All systems are quad-core, and use hardware preprocessing.

• Onsite configuration: Dell Precision workstations with Osprey 230 capture cards
• Offsite configuration: Dell 2950 with Digital Rapids capture cards.

Software

The encoders are running Windows XP and use Windows Media Encoder, with my recommended tweaks and registry key settings applied.

Codecs

• Video: Window Media Video 9 Advanced Profile (aka VC-1 Advanced Profile)
• Audio: Windows Media Audio 9.2

The audio was originally going to be WMA 10 Pro for improved efficiency, but we fell back to WMA 9.2 in order to have Silverlight 1.0 compatibility. Fortunately the audio feeds are either mono or might as well be, so we can win some efficiency back by encoding in mono.

300 Kbps streams

• Video: 480x360, 29.97 fps, 259 Kbps
• Audio: 44.1 KHz mono, 32 Kbps

500 Kbps streams

• Video: 480x360, 29.97 fps, 442 Kbps
• Audio: 44.1 KHz mono, 48 Kbps

First, install it already! It can be downloaded here, and probably most of you picked it up via Microsoft Update by now. James Clarke had a great overview blog post about it which I recommend.

This first post I’m going to focus on Smooth Streaming and multibitrate streaming in general.

“Adaptive Streaming” and Smooth Streaming

Here’s the big one: Expression Encoder can now do multibitrate encoding. Alex Zambelli has a good history of the technology.

When doing multibitrate encoding, EE does multiple simultaneous encodes from the same source file. While the big driver for that feature is Smooth Streaming (another overdue blog topic), we also support encoding a single multiplexed “Intelligent Steaming” WMV file or individual WMV files for each bitrate.

SmoothHD.com

And If you haven’t seen it yet, head on over to the SmoothHD.com demo site we do with Akamai’s  and check out the technology in practice. And yes, all those files were encoded with SP1.

SmoothHD.com in action. Those bars in the lower right show which data rate band you’re getting; mouse over for more details. Click on it to bring up a diagnostic menu and play around with scaling. And boy, that 20 Mbps DSL upgrade I’ve got on order can’t come soon enough!

Previewing Smooth Streaming with Expression Encoder

So, you want to play around with Smooth Streaming, but don’t want to have to upload your files to Akamai every time you want to do a quality check? Fair enough, but we don’t have media players that support the Fragmented MPEG-4 format used in .ismv files yet. Fortunately, EEv2 SP1 also includes a little localhost Smooth Streaming web server built into it. If you encode to a Silverlight 2 template and make sure that “Preview in Browser” is checked, you’ll get Smooth Streaming working, heuristics and all! It’s only on the local machine, and only when EE is running, but it’s a start.

To preview the video quality of Smooth Streaming, you can also make individual WMV files and watch them as normal. The output bitstreams are identical no matter what the file is wrapped in.

WMSnoop remains a great (and free) way to check out the GOP structure of WMV files. Below (click to zoom) you can see how Smooth Streaming produces a rock-solid cadence of a keyframe (in yellow) every 60 frames, while still being able to insert extra keyframes as needed for high quality scene changes.

Encoding for Multibitrate

Unlike the old FSDK-based MBR encoder, each stream is encoded in its own thread, which means a whole lot of cores can be saturated; my 8-core typically is running at nearly 100%, and if HD is involved, a 16-core would as well. The limiting factor can wind up being the speed of the source decoder past a certain point. Here’s a pretty typical load I’ll see when running from a Lagarith source file.

Multibitrate Video Options

Most of the parameters must be identical per stream; only frame size and bitrate can vary each. Other parameters like frame rate, audio, and codec settings are fixed. This is to facilitate Smooth Streaming by making sure that I-frames appear at the same point in each output stream. For technical reasons with the current VC-1 SDK, this also means only 1-pass CBR is supported for multibitrate encoding.

Multibitrate Audio Options

Only one audio option can be selected when doing Adaptive Streaming encoding. This isn’t the limitation it would have been in the WMA days, as WMA 10 Pro’s low bitrate audio quality is so good. We find 64 Kbps provides great quality for most content, and fits into most web delivery bitrates.

Multibitrate Container Options

There are three container choices

• ASF Single File. This gives you one WMV file for each video data rate, each including its own copy of the audio. This is an easy way to make multiple versions of the same source file.
• ASF Multiple File. This gives a single WMV file containing all copies of the video and one copy of the audio, ala Intelligent Streaming. This provides bitrate switching when used on Windows Media Services.
• IIS Smooth Streaming. This is for use with the forthcoming IIS Media Pack version that enables Smooth Streaming. To use this today, you’ll need to be part of Akamai’s beta program. That gives one output file per stream. There’s no way at the moment to play these files easily on the desktop, as they use the Fragmented MPEG-4 file format, which isn’t broadly supported in media player software; it’s mainly used inside of set top boxes like a TiVo.

Multibitrate Advanced Codec Settings

There’s a few required settings to get optimal Smooth Streaming compatible content in EEv2SP1. Smooth Streaming requires that all bitrates start have Closed GOPs (Group of Pictures – a keyframe and all the frames that reference it) starting on the same frame every few (typically 2) seconds. This means that the decoder gets a continuous sequence of frames to decode without any overlaps or missing frames, and without having to run two simultaneous decoders. These are compatible with (but not needed for) ASF, and using them gives the option to remux to Smooth Streaming at a future date.

• Maximum QP=31. This is a new feature in SP1, which specifies the maximum allowed Quantization Parameter that can be used. This is similar to the old WME “Smoothness” control which determines the tradeoff between preserving image quality versus frame rate. We’re trying to avoid a frame getting dropped in one data rate but not another, which could throw off GOP alignment. QP 31 is the most compressed a frame can be in VC-1, thus minimizing the chance of a skipped frame.
• Adaptive GOP=Off. Adaptive GOP tells the codec that it can “reset” the Key Frame Interval, making it into really “keyframe at least every.” That improves efficiency, but makes it possible that the different output bitrates might wind up starting new GOPs at different times, without any mechanism to resync them. When Adaptive GOP=Off, it’ll still insert an I-frame where appropriate to optimize quality, mainly when there’s a video cut in the middle of a GOP.
• Closed GOP=On. In a Closed GOP, the first frame of the GOP is always an I-frame (keyframe), and no frame in the GOP references any other GOP. That makes each GOP fully self-contained, as required for Smooth Streaming. If Closed GOP were off (Open GOP), B-frames can reference frames in other GOPs. This helps efficiency a little bit, but means you always need to have decoded most of the previous GOP to play the first frame of the current one. Even if Closed GOP=On, any additional I-frames inserted due to scene changes may be an Open GOP for improved efficiency.
• Output Mode: Elementary Stream Sequence Header. With an Elementary Stream Sequence Header, the VC-1 bitstream includes all the data the decoder needs to play back the chunk of video in that chunk. It’s required (and it’s a tiny bit of data; there’s no downside to using it).
• Insert Skipped Frames=On. With Skipped Frames, a frame doesn’t get encoded due to exceeding the Max QP or because it’s just a static frame without motion, consist of just a flag indicating “I’m the same as the frame before.” If Skipped Frames=Off, the frame before the dropped frame has its duration extended to cover the missing frame(s). The actual video playback is identical in either case; but this  ensures dropped frames don’t cause a GOP misalignment. There’s no real downside to using this for all files, Smooth Streaming or not.

Tips for multibitrate encoding

General Codec tweaks

These are the changes I made from the Adaptive Streaming default, implementing the typical “make it look good” settings I recommend for EE in general also apply to multibitrate compression. Just don’t mess with the settings up above!

Always On recommendations

These settings won’t have much impact on encode time, but can help quality significantly, particularly with

• Adaptive Dead Zone=Conservative . This tells the codec to bias toward softness over blockiness as a frame gets more compressed.
• Overlap=On. The Overlap filter softens the edges of blocks, reducing blocking and artifacts at higher compression rates. Web delivery is almost always at rates where the Overlap filter helps quality.
• Scene Change Detection=On. Scene Change Detection tells the codec to buffer ahead 16 frames to look for fades, flashes, and scene changes, and dynamically change the frame type based on that. This particularly helps with fades to/from black, and with flash frames (think raves and gunfights). While it can insert an I-frame, it won’t disrupt the pattern locked in by having Adaptive GOP=Off. Scene Change Detection never hurts quality, and often helps. Scene Change Detection maps to the Lookahead parameter used in the Format SDK.
• Search Range=Adaptive. Lets the codec dynamically adjust the search range based on the amount of motion in the content. Helps quality a lot when there’s more than 64 pixels horizontally or 32 pixels vertically of motion between two P-frames, without spending CPU cycles on frames with more sedate motion.

Quality over Speed recommendations

These are settings I use when I’m more worried about optimum quality than the speed of encode.

• Complexity=4. Complexity controls how hard the codec works on each frame, particularly the precision and thoroughness of looking at motion from frame to frame. Complexity 3 is a great default, since it provides most of the value of 4 and 5 at a higher speed; quality drops of quickly as you go to 2 on down. But 4 is a little better yet, particularly with complex motion like particle effects. Complexity 5 is overkill most of the time, but I wind up using it sometimes when I’m feeling more fussy than rushed, “just in case.”
• Chroma Search=Full True Chroma. Chroma Search looks for where the chroma (color) of the image changes differently that its luma (brightness).. This is particularly helpful with motion graphics and animation. Chroma is only a third of the data in the encode, so a full precision chroma search doesn’t add all that much to the encoding time, and with the right content can really help squeeze things down.
• Match Method: Adaptive. This switches between using Sum of Absolute Differences (SAD) and Hadamard motion matching on a per-macroblock basis. Don’t sweat what that means too much; just know it’s slower and a little faster than the default SAD-only. Full Hadamard is slower yet and sometimes lower quality than Adaptive.

Stretch, not Letterbox

Expression Encoder defaults to letterboxing to be used when the output frame size doesn’t match the source aspect ratio. However, even a slight variation in output aspect ratio from the source’s (typically to get the encoded height and width divisible by 16, the most efficient option) can result in a thin black line of letterboxing (top and bottom) or pillarboxing (left and right). If Resize Mode=Stretch and Mode=Profile Adaptive, Profile Adaptive will get the output frame size in the right ballpark while Stretch adjusts the video slightly to avoid those darn black lines. To make sure the output aspect ratio really is perfect, you can set the Video Aspect to that of the source, and on playback it’ll stretch the image pack to the exact original frame size and shape. Leaving it at the default of Square Pixel generally looks fine as well.

Note that you may need to reapply the Profile Adaptive setting if you apply a job preset to multiple files at once. That can reset to Mode to Custom.

Integer Framerates

By default, Adaptive Streaming profiles gives a keyframe every 2 seconds. However, that’s a precise 2.0000 seconds. So, if your source is a typical NTSC frame rate like 29.97 or 23.976, you’re 0.1% short of 30 and 24 frames a second, and so wind up getting a keyframe every 47 or 59 frames, not the 48 or 60 you might expect. If you set the frame rate to the whole number equivalent, the encode will actually maintain the source frame rate correctly (the value serves as an upper bound, but EE won’t insert duplicate frames when set higher than the source). But it will round up the keyframe cadence to the whole number, so you’ll get the keyframe every 48 or 60 frames like you’d expect.

Not really a big deal in practice I suppose, but I’ve long since internalized the multiples of 48 and 60, which makes it easier to me to figure out where my keyframes are.

Determining bitrates

Picking the optimum single frame size and data rate combination for a particular piece of content can already be a challenge. Having to determine optimal sizes for a bunch of different data rates. Fortunately, Alex Zambelli is as always ready with a handy utility for us – the Smooth Streaming Multi-Bitrate Calculator!

All you need to know is

• What bitrate works for the highest frame size for your content
• What minimum bitrate you want to go to
• How many steps to take

There’s actually some pretty deep math in there that knows how VC-1 bit-per-pixel requirements change as frame size changes, etcetera.

To whet the appetites of my compressionist friends, one much-requested new feature in the Format SDK .dll in Windows 7: It now can go up to 8-way threading, compared to the 4-way threading introduced with WMP 11 and Vista! This should nearly double encoding speed on 8-core machines doing HD encoding.

The number of threads the encode gets is determined by the height of the video frame.

• 1-119 lines 1 thread
• 120-239 lines: 2 threads
• 240-479 lines: 4 threads
• 480+ lines:  8 threads

This is because each thread gets a slice of the video frame, so more threads mean that vertical motion gets a little less efficient to encode. And as always, the number of threads used can be overrridden with Alex Zambelli’s WMV9 PowerToy. Its looks like we’ll get one last update out of him after all to add “8” to the drop-down list.

This new mode will work with any program that uses the Format SDK .dll, including Windows Media Encoder, the new Windows Live Movie Maker beta, and Virtual Earth. It won’t apply to VC-1 Encoder SDK based apps like Expression Encoder 2, but obviously we’ll want to add this to the VC-1 Encoder SDK as well.

So, if you’ve got an 8-core processor, or a 4-core with hyperthreading, give the Win 7 beta shot. I’d love to hear from anyone who gets some comparisons for encoding time between XP/Vista and Win 7.

A fellow called Kurtnoise13 jumped on it on the Doom9 Forums discussion about the SDK. He quickly created a new version that supports AVISynth files as source as well. The syntax for a simple PAL 1.2 Mbps encode (from the ReadMe) looks like:

AVS2ASF -i VideoWithAudio.avs -o file.asf -framerate 25 -rate 1200 -profiletype 2 -bframes 2 -maxkeydist 6 -vbv 300000 -peakrate 2000 -pr AudioEncode.prx

Be warned that while the syntax looks a lot like WMCmd.vbs the details can be deceptively different. In particular, units can be different (Kbps instead of bps makes a big difference!). Also, since the VC-1 Encoder SDK is video only, a PRX file defining the audio setting also has to be used; the Format SDK is called to encode audio and mux to WMV.

AS2ASF also supports elementary stream output, so it’s able to generate a .vc1 file for Blu-ray authoring.

For those not familiar with it, AVISynth is an open source frameserver that dynamically processes video and lets it look like any DirectShow source to other tools. This means that intermediate files aren’t needed – an AVISynth “.avs” script can define the source and all the processing to apply to it. And there’s a huge community of AVS plugins to handle new format times and processing modes; almost any kind of file out there can be handled by AVISynth, and almost any kind of processing you’d like to apply is available. For a simple example, the below script

• Opens a PNG sequence at 24 fps
• Associates a WAV file with it
• Crops it from 640x360 to 640x352
• Outputs it in YV12 (8-bit 4:2:0)

a=imageSource("D:\Elephant's Dream\PNG 360\%05d.png",1,15691,24)
b=wavsource("D:\Elephant's Dream\ED-CM-St-16bit.wav")
audiodub(a,b)
crop (0,4,0,-4) # crop height 352
ConvertToYV12()

Anyway, I wanted to highlight this technology. The source code is available as a Visual Studio project, and I’d love to see other members of the community continue to enhance it.

A few things that’d be cool:

• A simple batch automation GUI
• A simple automatic AVS generator, so a .avi or .mov file could be specified as source and the right AVS to wrap it could be build on the fly.

Bear in mind that Windows Media Encoder is older than five years now, which is when multimedia products start moving from Mainstream Support to Extended Support in the support lifecycle. We don’t set support policy for unreleased products, so note that there’s been no formal guidence on whether WME will be supported on Windows 7.

Hotfixes

WME is ancient code at this point, and all updates since the original Windows Media 9 Series launch have been done as hotfixes. You need to have these installed for security, stability, and performance.

Vista compatibility hotfix

If you’re on Vista, you’ll absolutely want to install this one.

FIX: You may experience issues when you use Windows Media Encoder 9 Series on a computer that is running Windows Vista

I covered this in my first non HD DVD blog post here on On10.net!

It can be a bit tricky to install sometimes, so follow these instructions from the invaluable

Windows Media and DirectShow tips, gotchas and code bits blog.

This hotfix address three issues:

Issue 1

When you run the Windows Media Encoder command-line script WMCmd.vbs, the script host Cscript.exe may crash.

Issue 2

The icons that appear on the encoder toolbar and in the encoder dialog boxes are displayed by using a low bit depth. Therefore, the icons appear to have a low resolution.

Issue 3

When you configure the encoding profile or start an encoding session, the encoder may crash.

Multithreaded Multiple bitrate encoding hotfix

This one fixes a pretty embarrassing bug – Format SDK 11 wouldn’t use multiple processors when encoding in multiple bitrates (aka “Intelligent Streaming”). Originally blogged here.

FIX: The WMV9 encoder multi-bitrate performance on a multiprocessor system is severely impaired

Critical Security Fix

This is a security fix for a critical vulnerability that could allow remote code execution on a machine running WME. Install it before you launch WME! Note that the vulnerability requires two things that shouldn’t be happening anyway on a production encoder

• WME is running logged in as an administrator
• The machine is being used to browse the web

Microsoft Security Bulletin MS08-053 – Critical. Vulnerability in Windows Media Encoder 9 Could Allow Remote Code Execution (954156)

Current Format SDK version

WME in the end is mainly a graphical front-end to the Windows Media Format SDK. The actual codec version you’re running is determined by the SDK. Most people get those updates bundled with Windows Media Player, so as long as you have the most recent WMP version for your platform, you’re good to go.

If there’s some reason that you don’t want to update WMP but still want the current codec .dlls, you can also install the Format SDK directly. The SDK version also includes lots of samples and all the normal SDK goodness for building apps for authoring Windows Media content.

For vendors of compression tools, it’s always a good idea to chain the redistributable installer (WMFDist11-WindowsXP-X86-ENU.exe) along with your installer, so you can ensure that users have the current version. That installer can also un manually to install .dll updates without updating .

I’ve seen way too many production encoders in the last few months that have WMP 9 and thus the 9.0 SDK. That’s a good six years old now, and will offer much lower quality and perf than the current versions deliver. Remember this comparison?

Windows XP, Vista, or Server 2008: Format SDK 11

Windows XP shipped with Windows Media Player XP, which was a flavor of WMP 7. However, the almost universally installed Service Pack 2 included WMP 9. WMP 11 is also available for XP, and should absolutely be installed on any XP-based encoder (and really all XP machines in general).

Windows Vista shipped with WMP preinstalled, so no action is required there.

On Windows Server 2008, Windows Media Player, and hence the FSDK, is installed as part of the Desktop Experience feature. You’ll want to enable the DEP on any encoder box (and in general for any 2K8 box you’ll be using as a workstation).

Windows Server 2003: Format SDK 9.5

The most recent version for Server 2003 is the older Format SDK 9.5 (which is still better than 9.0). As service to our customers looking for a high degree of confusion, Format SDK 9.5 was released with Windows Media Player 10.

Note that running on Server 2003 means you won’t have access to the SMPTE-compliant “WVC1” flavor of Windows Media Video 9 Advanced Profile, nor the excellent WMA 10 Pro LBR audio codec, and hence the 32-96 Kbps range for WMA Pro. The codec is also only 2-way threaded instead of 4-way threaded, slower in general, and lacks support for Tarari acceleration and the advanced registry keys.

If you need to encode on a Server OS, you should use a VC-1 Encoder SDK based product or upgrade to Server 2008 (ideally both!), which will give you better video. However, WMA 10 Pro will still not be available on Server 2003.

There have been reports of people using the WMFDist11-WindowsXP-X86-ENU.exe installer from FSDK 11 set to XP compatibility mode to install the FSDK 11 .dll files onto Server 2003. While there aren’t known issues with this, I should point out that this isn’t a supported configuration for Server 2003, and so Microsoft support will not be available for any issues when running with mismatched .dll files.

64-bit Windows: Format SDK 11 (32-bit)

While there is a “Windows Media Encoder 64-bit edition”, don’t use it (I’m not even going to link to it). It predates FSDK 11, and offers lower quality and performance than FSDK 11 does; the performance advantage of running 64-bit native code is a lot smaller than the other improvements in FSDK 11.

So, keep using the 32-bit stuff on 64-bit as well as 32-bit versions of Windows.

Registry keys

Once upon a time, this blog seemed to be mainly about using special registry key options in the Format SDK. Thankfully, with Expression Encoder and the other VC-1 Encoder SDK based products mean we have GUI access to all of these now in modern products.

First off, install Alex Zambelli’sWMV9 PowerToy, which is a simple .NET GUI for setting the codec settings. The tooltips are a better reference than anything I’ll type here; I’ll just give some basic recommendations for different scenarios.

“Always On” choices

This is a good set of defaults with no significant performance downside that almost always helps quality.

Perceptional Option: Adaptive Dead-Zone 1. This maps to the “Adaptive Dead Zone: Conservative” option from Expression Encoder. This lets the codec reduce detail before introducing artifacts, and generally improves quality at lower bitrates.

In-Loop Filter: On. This turns on the In-Loop deblocking filter which softens the edges of block artifacts. This improves the current frame, and also future frames based on it.

Overlap Filter: On. This further smoothes the edges of blocks. It can reduce detail a little at high bitrates, but is almost always helpful at typical web bitrates.

B-Frame Number: 1. Turns on B-frames, and hence enables flash/fade detection when using Lookahead or 2-pass encoding, and also improves compression efficiency.

Lookahead: 16. Tells the codec to buffer ahead 16 frames in 1-pass (CBR or VBR) encoding, letting the codec detect flash frames and fades and switch the frame type based on it. Maps to the “Scene Change Detection” option in Expression Encoder. It will increase end-to-end latency by that many frames in live encoding, but is generally worth it due to quality improvements.

High Quality Live

Assuming you have a fast enough machine to run these settings in at least Complexity 3, they will improve the live experience.

Motion Search Level: Fixed Integer Chroma. This adds basic chroma search to the encoding, which can help the quality of motion graphics significantly. It’s a pretty small boost with more typical camera-shot content, so turn it off if you have perf issues; getting to Complexity 3 is more important.

Motion Search Range: Adaptive. This tells the encoder to switch to a bigger motion search range for frames with high motion, and then go back to a smaller range when motion dies down. This dramatically improves quality with higher motion at bigger frame sizes. The default range is 64 pixels left/right and 32 pixels up/down, so if any objects move more than that between any two P-frames (if you’re using B-frames, that’ll be 2 frames apart), than this feature will help.

High Quality Offline

These settings offer maximum quality for offline encoding, and are slower yet. Use them when you’ve got the time, and run at Complexity 4. Complexity 5 ignores the Motion Search Level and Motion Match Method settings, making it lower quality. The only time you need to use Complexity 5 is if you can’t set registry keys (in which case it’s quite a bit better) or need to use WME to scale your video, as scaling quality is much better in Complexity 5. However, you’re better off preprocessing in another tool if at all possible.

Motion Search Level: Fixed True Chroma. This is a full-precision motion search for chroma. It never hurts, and can help quality a lot with motion graphics and animation.

Motion Match Method: Adaptive. This switches between the Sum of Absolute Differences (SAD) and the Hadamard method to compare motion between frames as appropriate for each macroblock. Full Hadamard can be higher quality for some very complex content, but the Adaptive mode is faster and better most of the time.

Use hardware preprocessing

WME’s preprocessing was designed for good live encoding performance on circa 2002 machines, which means it’s tuned far more for speed than quality. In particular, deinterlacing and scaling aren’t very good (to say the least). So if you’re using WME with a hardware capture card like an Osprey, you’ll get better results doing all the preprocessing on the card, and just passing the final scaled, cropped, deinterlaced YV12 bitmaps off to WME. This also saves some additional CPU cycles for the codec. For file-to-file encoding, you also want to do any deinterlacing and scaling before you import the file into WME. Expression Encoder has great scaling and good deinterlacing, so that’s another reason to use it over WME.

Optimum hardware

If WMP 9 is all you have on your encoder because that’s all there was when you configured it, you’re going to be running on some very old and very slow hardware! Bear in mind that Moore’s Law predicts you can get twice the computer for your dollar every 18 months or so. So today’s best machines will have about 16x the encoding horsepower of what you could have had when WME was released!

Encoder Complexity

When tuning hardware, the goal is to enable higher encoder complexity values to be used. Your target is at least complexity of 3; that’s most of the way towards optimal quality; lower values sacrifice a quite a bit of quality for improved speed, while higher values sacrifice a lot of speed for only a little additional quality improvement. Complexity 4 is optimum when time/performance isn’t a concern. As mentioned above, Complexity 5 in FSDK 11 invalidates some registry keys (not an issue with VC-1 Encoder SDK).

For live encoding, the best way to test is to run the encode using real-world sources and make sure you’re not dropping frames at your target frame size. You’ll want to set Video Smoothness to 0 for this test to make sure frames aren’t being dropped for quality reasons instead of performance reasons.

CPU

Since the encoder is 4-way threaded, you want to be on at least a 4-core machine. These are dirt cheap these days, with affordable 4-core laptops coming soon. Generally speaking, a 4-core is a lot faster than a 2-core at a slightly faster clock speed.

Clock speed is a big factor as well, but it only matters in context to the generation of processors. For example, the new Intel i7 “Nehalem” get a whole lot more work done at 3 GHz than the original P4 “NetBurst” CPUs did at 3 GHz.

Memory

Memory isn’t a big factor in encoding, as long as you have enough. You never want an encoder to start swapping, as performance will fall through the floor. 2 GB should be plenty for a dedicated encoder, even doing HD encoding.

Tarari Encoder Accelerator

If you’re encoding to higher resolutions and just can’t hit the performance you need at a sufficient quality, check out the Tarari Encoder Accelerator. This offloads a number of the computationally intensive parts of compression to a PCI-X board. While it can’t produce the absolutely best quality a tweaked Complexity 4 + registry key software encode can, it can certainly deliver much higher quality for live HD than pure software, particularly on older computers.

The Tarari board is also fully supported by VC-1 Encoder SDK products, so your investment isn’t tied to a FSDK workflow.

Move to a VC-1 Encoder SDK based product!

So, that’s how to get best results out of legacy Format SDK encoders, particularly WME. But really, it’s time to start planning a migration to a VC-1 Encoder SDK based encoder. If there’s something you need that they simply don’t provide, please let me know. Future improvements in VC-1 and Windows Media encoding are going to focus on the SDK, so the gap between Windows Media Encoder and the best encoders is going to be growing bigger over time.

Extra – rollup of Windows Media Encoder related posts

And just so they don’t disappear off the bottom of the blog, here are my blog posts about WME and the FSDK 11 codecs that readers may still find useful:

Low Latency webcasting with Windows Media and Silverlight

Compression for Silverlight Webcast posted

New WMcmd.vbs with improved multithreading and new presets

http://on10.net/blogs/benwagg/New-PowerToy-and-registry-key-documentation/

Hands-on with the Temporary Insanity trailer

http://on10.net/blogs/benwagg/Encoding-for-the-Zune/

http://on10.net/blogs/benwagg/Elephants-Dream-720p--2-Mbps/

http://on10.net/blogs/benwagg/My-Live-Webcasting-with-Windows-Media-presentation-from-Streaming-Media-East/

http://on10.net/blogs/benwagg/My-Mix-07-session-about-WMV-Encoding-and-Expression-Media-Encoder/

It’s full of awesomeness, but I wanted to excerpt the section describing their 1st gen, 2nd gen, and HD encoding settings and workflow.

First Generation Encoding

Our first set of encodes are based on WMV3 and WMA in ASF with WMDRM10 (Janus). We chose these standards because the Janus components have been widely adopted by our CE partners such as Roku, LG Electronics, Samsung, TiVo, and of course Microsoft Xbox.
We encode most content at 500, 1000, 1600, and 2200kbps VBR, but some titles whose source quality merits it have also been encoded at 3400kbps. The highest bitrate encodes are fit into 720x480 non-square pixels (the usual 1.2 PAR for widescreen content, 0.9 PAR for 4:3), but optimum encoding at lower bitrates is achieved with fewer pixels. Encoded films are normally at 24fps to match the source, while shot-to-video and mixed material is de-interlaced to 30fps (or 25fps for PAL content).

Netflix has been using anamorphic video all along, which I think is an underused feature of Windows Media and other formats. When you’re limited to 720x480 pixels, you want to encode all the pixels you’ve got, without having to synthesize any extra.

Second Generation Encoding

The new Silverlight player (that some users are helping us test as I write) uses VC1 Advanced Profile encoding with PlayReady DRM. A key property is that each GOP header includes frame size and resolution, which allows us to assemble a stream on the fly from different bitrate encodes as your broadband bandwidth fluctuates. (Another key feature is more coverage, including Intel Macs and Firefox users.) We expect to switch completely to the new player later this year.
The VC1 encoders are more efficient than the WMV3 encoders, so we are currently encoding VC1AP at slightly lower birates: 375, 500, 1000, and 1500kbps, all square pixel. At some point we are likely to add a couple more resolutions of non-square pixel encodes capturing the original pixel-aspect-ratio of the source.
We are also re-wrapping the VC1AP encodes in WMDRM10 for CE devices, which will gradually switch to the more efficient encodes in future firmware upgrades.

This is a great example of the improved efficiency of the VC-1 Encoder SDK and tools based on it. Not all Windows Media/VC-1 encoding is equal; the latest tools can offer a very meaningful reduction in bitrate required for a give quality level, improving user experiences and the cost of delivering the content.

The new encodes are backwards compatible to older hardware and software encoders, so even older devices can take advantage of the improvements.

High Definition Encodes

Today we have rights to deliver about 400 streams in HD (720p). More titles will be added over time. We experimented with first-generation WMV3 encodes at 4000kbps and 5500kbps, but settled on second-generation HD encodes with VC1AP at 2600kbps and 3800kbps, which extends their accessibility down to lower home broadband connections. As with SD, encodes of film material are at 24fps, and encodes of shot-to-video material are at 30fps (or 25fps for PAL), rather than the 60fps that would come from a Blu-ray disc - we judged the 60fps content as too expensive of bandwidth for now. In general, these encodes are definitively better than SD, but won't challenge well-executed Blu-ray encodes - that would require a bitrate out of reach for most domestic broadband today. We believe Moore's law will drive home broadband higher and higher enabling full 1080p60 encodes in a few years.

There’s codecs for you – a good 720p experience at 3800 Kbps, which is probably a little below the average 480p bitrate used for MPEG-2 on the DVDs that fly around the nation in those Netflix envelopes.

I’m not that worried about 1080p60 encoding myself. The vast majority of fictional content is shot 24p, including 99% at least of Blu-ray discs. While Blu-ray players may output 1080p60, the encodes, and hence bitrate requirements, are still 1080p24. I find that typical film content in VC-1 wants around 6-8 Mbps for a 1080p24 experience enough better than 720p to be worth the trouble.

I just got my Xbox set up to handle the Netflix streams – now I’ve got to go check out some of those 400 HD titles! Hmm. Pan’s Labyrinth and Heroes.

Anyway, it’s great stuff. I love it when partners can share this kind of detail about what they’re doing.

• There's a new scaling algorithm that's a lot faster and much higher quality - a nice compromise when you can get it!
• Additional optimizations have been done for the VC-1 video decoder, so playback will be faster. The gains are biggest for content using B-frames.

Since the default Expression Encoder templates have "always on" scaling and its presets use B-frames by default, the above will provide a nice performance boost for existing content. The above are "always on" features - you don't need to update anything to take advantage of them. So expect smoother frame rates on lower-end machines.

Further information:

• Silverlight 2 breaking changes .doc file
• Silverlight 2 ReadMe
• ScottGu's Blog post on RC0

EDIT: Added ScottGu link

• A new Silverlight 2 base player using .NET. Among other things, this will enable players that display the video at 100% scale by default, improving quality and performance.
• New A/B compare modes. Awesome stuff for high-touch encoding and codec tweaking
• Some even more VC-1 advanced options for High Codec Nerditry.
• The first public release of Microsoft H.264 compression technology.

Anyway, a lot of good stuff for a SP1. I'm looking forward to it.

The big news is that, yes, we're going to add support for MPEG-4 to Silverlight, in the version coming after the fast-approaching fall release of Silverlight 2. Specifically, this will be H.264 for video and AAC for audio.

Why?

So, why MPEG-4 support in Silverlight? It's pretty straightforward, really. We have customers with libraries of H.264 content they wanted to publish to Silverlight, but didn't want to reencode to VC-1. Silverlight's strengths go far beyond media playback, and customers wanted the choice to deploy a wide variety of existing content within Silverlight. Silverlight aspires to provide as much choice as feasible as to how Silverlight can be authored and delivered. H.264 support is something we'd considered for past versions, but there were higher priority features we needed to deliver first. Silverlight 2 provides us a very rich base for delivering web apps, so we can start spreading our wings a bit.

What?

We've got more technical details we'll be sharing at IBC and later, but today, I'll just quote the details from Scott Guthrie's interview:

PressPass: Will you be showing or announcing anything new at the IBC conference this week?

Guthrie: Yes. At IBC 2008 we will be demonstrating a technology preview of H.264 video and Advanced Audio Coding (AAC) audio playback support in Silverlight, and H.264 authoring using Microsoft Expression Encoder and Windows Server 2008 for delivery. Until now, Silverlight has supported the SMPTE VC-1 and Windows Media formats, as well as MP3 for audio, enabling customers to take advantage of broad support across the Windows Media ecosystem, including third-party tools, service providers and content delivery networks.

We’ve always wanted Silverlight to support a variety of formats, so today we’re announcing that H.264 and AAC support will be available in a future version of Silverlight, which will offer content owners greater flexibility and choice to deliver video and audio.

PressPass: Historically, people have associated Microsoft with VC-1. Does this signal a change in direction?

Guthrie: No. Although we have been working with VC-1 for some time, it’s not widely recognized that Microsoft has also been an active participant in the standardization of H.264/MPEG AVC for many years, and we’ve included H.264 support in several Microsoft products. Microsoft’s Gary Sullivan was the chairman of the Joint Video Team (JVT), which developed the H.264 standard, and he recently accepted an Emmy Award on behalf of the JVT.

PressPass: Does this mean that Silverlight is moving away from Windows Media?

Guthrie: Not at all. This is about offering our customers more choice. Media producers and distributors around the world have enjoyed the high quality, flexibility and affordability of Windows Media formats for over a decade. As a testament to its pervasiveness, Windows Media can be found in almost every conceivable media scenario from desktop home video to feature films and TV broadcasts.

Again, don't read this as as a big change around our strategy for media formats. We've long-supported WMV and MPEG-4 side-by-side in products like Xbox and Zune. As I posted last week, Microsoft has been contributing to H.264 since its inception.

This doesn't represent any change in our support for Windows Media. Windows Media continues to work well for today's Silverlight customers. I expect (not a goal, just a prediction) that the majority of Silverlight content will remain in WMV well after we release MPEG-4 support.

In particular, VC-1 will likely remain the codec of choice for HD for some time. Comparing VC-1 Advanced Profile to H.264 High Profile with all the bells and whistles turned on, VC-1 only needs about half as many MIPS per pixel for playback. This won't matter as much for lower resolution content, or podcasting stuff that's in the simpler Baseline profile, but makes for a big reduction in system requirements for 720p and higher.

Where/When can I learn more?

If you're at IBC, swing by! If you want a head start on using MPEG-4 in Silverlight, feel free contact me directly. And we'll have plenty of more technical info to share down the road a bit.

So, I was really excited that our own Dr. Gary Sullivan, chairman of the Joint Video Team, was on hand last week to receive a 2008 Primetime Emmy^® Engineering Award for developing H.264/MPEG-4 AVC High Profile. In the words of the Academy:

Joint Video Team Standards Committee (JVT) for the development of the High Profile for H.264 / MPEG-4 AVC. The High Profile enables high definition images in the H.264 video coding system used today to deliver HD video over satellite and cable TV as well as Blu-ray Disc. The JVT is a standardization team comprised of members from the International Standardization Organization (ISO), the International Electrotechnical Commission (IEC), and the International Telecommunication Union (ITU).

The award recognized Broadcom, NTT DoCoMo, Dolby Laboratories, Fraunhofer Heinrich Hertz Institute, Microsoft, Motorola, Panasonic, Sony, and Thomson as contributing companies, as well as standards organizations ISO, IEC, and ITU.

Gary has long represented Microsoft as as chair and co-chair of the Joint Video Team that coordinated the H.264/MPEG-4 AVC effort between ITU-T VCEG and ISO/IEC MPEG, amongst his many other efforts like DXVA; his page on Wikipedia has more details.

I suppose some people might be surprised at Microsoft's inclusion here, but they shouldn't be. While Microsoft has a long heritage of making our own ground-up codecs like VC-1, we've also long been involved in standards-based codecs, including the original MPEG-4 Part 2 as well as MPEG-4 Part 10/H.264. And we've got a number of products that incorporate H.264 today, including Mediaroom, the Zune 4/8 and 80, and Xbox 360. H.264 are and VC-1 are both great codecs, with somewhat different sweet spots, and we use either or both as appropriate in our products, platforms, and services.

The picture above has, left to right: Malcolm Johnson (director of the ITU standardization bureau), Ajay Luthra (JVT vice-chair from Motorola), Gary Sullivan (JVT chair from Microsoft), Thomas Wiegand (JVT vice-chair from Fraunhofer HHI), and Scott Jameson (chair of ISO/IEC JTC 1)

And a few other articles/posts about the award:

ANSI

EE Times

Dolby

Minoru "Mick" Etoh's blog (who represented DoCoMo at the ceremony)

What I haven't talked a whole lot about in too long is our support for authoring Silverlight video experiences on the Mac. While our own Expression Encoder 2 is Windows-only, we work with partners to provide WMV and VC-1 encoding on other platform. Telestream has been a great partner here, with their Flip4Mac QuickTime component and their Episode stand-alone compression tool (originally known as Compression Master, and which came along with their acquisition of Popwire).

Flip4Mac is a testament to both the portability of the Windows Media technologies and the flexibility of Apple's QuickTime architecture. It works as a QuickTime component, enabling QuickTime to play Windows Media content like any other supported format, with full access to all of QuickTime's features like hardware accelerated full-screen playback (all supported in the free version). For example, in the QuickTime Pro Properties window:

Looks just like any other QuickTime file. Even the ASF metadata shows up.

Flip4Mac also supports content authoring in the paid version, as well as import into content creation apps like Final Cut Pro. It's got pretty deep control, even exposing features like B-Frames that'd take a registry key with Windows Media Encoder.

So, how about a quick walkthrough through the Flip4Mac encoding settings?

Flip4Mac is accessed like any other exportable format from QuickTime (like MPEG-4). It also comes with a bunch of presets for typical scenarios.

It has the normal set of basic video export features you'd expect, nicely Macified. I appreciate the "Size: Current" so I don't have to always type in the frame size of the source when I'm not scaling.

The Advanced button reveals some deeper options, including a complexity control for speed/quality tradeoffs and B-Frame distance. You can also set Flip4Mac to deinterlace interlaced sources, or pass interlacing on through to the final encode. Since it can both encode and decode interlaced VC-1, Flip4Mac can be a great way to transfer editable 480i video sources over slow connections. It can match DV quality in 20% the data rate.

For Silverlight audio, Flip4Mac supports both WMA 9 Standard and 9 Professional. It doesn't encode the 32-96 Kbps WMA 10 Pro low bitrate modes that Silverlight 2 will be able to decode, but does have the full variety of WMA encoding modes.

And the coolest new feature of Flip4Mac: built-in Silverlight templates! This lets you build a complete Silverlight player (using Expression Encoder templates we provided) directly from any app that support QuickTime export.

EDIT: A correction was made; turning off Fast Cache in Windows Media Services does not have any effect on webcasting latency.

So, Streaming Media is doing a special "Europe edition" of Streaming Media, and I'm doing an article about webcasting for it. I've getting a bunch of questions about how to deliver low-latency live streaming to Silverlight, and so with their permission, I'm excerpting this section on that topic.

While by default, Windows Media can offer 15-20 second end-to-end delay, it's possible to drive it down to 2-3 seconds with best practices on a good network, and we're looking at what we can do to push it to below even that. The critical thing is to tune the encoder, server, and player latency together.

There's an older but more detailed "Reducing Broadcast Delay" document on this topic over at the Windows Media portal that may be worth perusing.

Low Latency Webcasting

Latency is the measure of how much time goes between when video enters the encoder and leave the user display. Latency is something that doesn’t matter at all in some markets, and matters a lot in others.

The reason we have latency is buffering, and the reason we have buffering is for quality and reliability. By having the server wait several seconds after a video stream is received before sending it out, it’s able to support more peaks and valleys in the data rate, and makes it possible for a dropped packet to be detected and resent before its needed. In the same way, buffering in the player lets it average out data rates and recover dropped packets as well.

Large buffers were extremely important in the modem era, and are still useful in many kinds of networks today. The defaults are good for delivering high quality content over a variety of networks. But when minimizing the latency is important, and reliable networks are available, the end to end delay can be reduced substantially.

Since total end-to-end latency is the sum of the encoder latency, server latency, and player latency, plus how long it takes the packets to travel between each of those, improving latency requires tweaks to the encoder, server, and player in parallel.

 

Encoder

On the encoder side, reducing buffer size reduces startup latency. WME and Expression Encoder both enable you to go down to one second.. Using Lookahead or Lookahead Rate Control will increase latency beyond the buffer value (typically about another half a second for LRC).

For audio, Windows Media includes the “WMA Low Delay” audio codecs from 192-64 Kbps which provide lower latency than the normal WMA modes. If you’re targeting Silverlight 1.0 or WMP 10 or earlier, you’ll want to use that for low latency. If you can require WMP 11 or Silverlight 2, you can use the lower delay yet WMA 10 Pro codec at 32-96 Kbps.

 

Server

WMS features like Advanced Fast Start can dramatically reduce latency for on-demand content, but don’t apply to live webcasting as the server has to wait for content to arrive from the encoder it in real time, and so can only play out at real time.

Server buffering can be turned off entirely in WMS for Windows Server 2003 and 2008, yielding a significant drop in latency.

EDIT:

The previous version of this post suggested that turning off Fast Cache would help webcasting latency. On further research, this turns out not to be the case. Disabling Fast Cache will slow down on-demand startup time, but will have no effect one way or the other for live content.

Player

Normally the player isn’t under control of the streamer. By default WMP dynamically picks an optimum buffer size based on its measurements of network and stream performance. However, it’s possible to lower the buffer size in the player’s options. This can help reduce latency when watching streams with a good connection, but could produce pauses in the video when watching video from the general Internet.

Silverlight makes the player buffer size a controllable parameter, so an optimal setting can be applied for the content, and even adjusted on the fly. This is controlled by the BufferingTime parameter in a Silverlight MediaElement.

Read the article yourself, but here's a few choice quotes from it:

• "NBCU plans to offer 3,600 hours of live programming from Beijing. That’s 212 live hours for each of the 17 days of the Olympics... In addition to the sheer volume of live content to be delivered—three times what was offered in 2004— what’s notable is that most of NBCU’s live programming—2,200 hours—will be delivered online at NBCOlympics.com."
• "The 2008 Summer Games in Beijing will mark the arrival of streaming as a viable alternative to the Olympics’ television broadcast. This summer, NBCOlympics.com will offer 4,400 hours of on-demand streaming in addition to its 2,200 hours of live programming, making the Beijing Olympics the most ambitious streaming media project in history."
• "To help meet that challenge, the NBCOlympics.com player offers a “metadata overlay” feature, which allows the player to display transparent data and navigation tools over the video window. This enables users to access statistics and other data without covering up, pausing, or leaving the primary video display. For example, play-by-play announcers’ dialogue can be keyed into an XML data stream, then rendered as a timecoded, scrolling text caption that transparently overlays the bottom of the video display. The player also enables the TiVo-like experience of pausing, rewinding, and replaying content, and these two features together allow viewers to use either the timecode or the play-by-play captioning to rewind to a specific point in the on-screen action and replay it."
• "A slew of DRC-Stream software and encoder boards from Canada-based Digital Rapids are being deployed in Beijing to populate NBCOlympics.com’s encoding farm, but other than committing to VC-1, NBCOlympics.com has yet to confirm encoding bitrates, frame rates, or frame sizes. (Without offering more specifics, Miller says NBCOlympics.com will be streaming through a managed bitrate solution to optimize the user’s connection, with a target maximum bitrate of 650KB/sec.) Digital Rapids is also supplying software to enable transcoding from other digital media formats into VC-1.Miller promises hundreds of hours of online HD video..."

Note that there's already a bunch of content up at NBCOlympics.com if you want to get an early taste of what's in store on 8/8/08.

To that end, I thought I'd do a demo to show how much improvement there's been in Windows Media since the launch of Windows Media 9 Series back in 2003.

A recent conversation over at the Streaming Media Advanced list sparked a rant from me about the importance of comparing implementations of codecs, not just codecs.

To that end, I thought I'd do a demo to show how much improvement there's been in Windows Media since the launch of Windows Media 9 Series back in 2003.

Below, you'll find two streams, encoded with the same settings but with tools from different eras. The streams are both

• Live encoding from a preprocessed file (so that preprocessing differences don't matter)
• And yes, it's the "Lady Washington" footage again.
• 640x360
• 29.97 fps
• 600 Kbps video
• 64 Kbps 44.1 stereo WMA audio
• 5 second buffer
• Keyframe every 5 seconds

The difference is that the first is encoded with the original Windows Media Video 9 codec (ala just Windows Media Player 9 installed, like a stock Windows XP SP2 machine), and the second with the VC-1 Encoder SDK implementation in Expression Encoder 2. Pretty dramatic differences, I hope!

...reminding me that I really need to blog how to make a Silverlight dual media player that can play two versions of the same clip in sync.

So, what makes this big difference? There's been a huge amount of work and three major releases (Format SDK 9.5 and 11, and VC-1 Encoder SDK) since then, so I won't give a complete list, but a few of the highlights.

• 4-way threading instead of 2-way threading, doubling performance on modern machines.
• Lots of SSE2 and SSE3 optimizations to improve performance.
• "Adaptive Complexity" that dynamically adjusts the complexity of the encoder, to make sure it's always using all available CPU power, without ever dropping frames. This compares to the old default live complexity of 1 (out of a 0-4 range).
• Using B-frames (supported in the old decoder, but not used in the original encoder) which improve compression efficiency and enable efficient encoding of flash frames.
• Lookahead Rate Control, where the codec buffers a few frames into the future, so it knows when it needs to start saving some bits for an upcoming keyframe, or when it's save to use a lot of bits on a few challenging frames.

A lot of the above are about performance. With offline encoding, better performance just makes for faster encoding. But for live encoding, it helps quality, because it allows the codec to do more math per pixel to find the optimum way to encode that file.

So, how big a difference is this? Below is a graph showing the Quantization Parameter (QP) for the two encodes. QP is a measure of how encoded each frame is, with higher values more highly compressed. In VC-1, the range is 0-31. A good rule of thumb is that QP of much below 8 looks pretty good, and QP of 8 or above...won't. Now, a live SD encode at 600 Kbps is pretty darn aggressive, so there's plenty of spots where both encodes certainly show artifacts. And quality varies a lot throughout the file as the the complexity of the video goes up and down, as this is a Constant Bitrate (CBR) encode. Note the relatively low QPs near the end of the file, where the easy credits scroll comes in.

But still, the modern VC-1 implementation (in red) with all of the above is dramatically better. While the old encoder (in blue) spikes all the way up to the maximum QP of 31, the new one is typically several QP lower, and maxes out at a QP of 20 instead.

That said, man, my Excel-fu sure has declined over the years. Hard to believe I used to teach classes on making good-looking Excel charts back in the early 90's. Anyway, just remember that lower is better, and red is our current stuff.

As I've said before, I feel that Expression Encoder 2 is the best compression tool at fulfilling the needs of its market since Terran Interactive's Media Cleaner Pro 3.1 (so long ago I can't even find anything on the web about it!). Here are some of my favorite things about it from my obsessive compression nerd's perspective.

• First desktop priced VC-1 Encoder SDK based product ($199).
• And the VC-1 implementation gives you great quality by default, and gives you an advanced mode that accesses the useful features, while handling the more esoteric ones behind the scenes. For example, the rule for WMV PowerToy was use Static Motion Vector Cost if using P-frame DQuant, otherwise use Dynamic. So Expression Encoder 2 just has a DQuant setting, and uses the right Motion Vector Cost based on that.
• High quality scaling and deinterlacing modes
• WMA Pro support, targeting Silverlight 2's support for that. So danceable music at 48 Kbps.
• "Smart by default" - you can use the same preset and feed it a PAL 16:9 interlaced file and a NTSC 4:3 progressive file, and it'll do the right thing without having to do a single click: you'd get a 640x360 deinterlaced 25p file and a 640x480 29.97 file.
• Which means you don't have a huge combination of settings to deal with all the video formats. Instead, the presets are scenario based; you just need one "1 Mbps streaming" option that'll handle all the video file variants.
• Great multi-clip templates, so you can make a video gallery, with each clip having its own optional thumbnail-based chapter navigation (hmm, that deserves a nice demo).
• Easy creation of thumbnail-based chapter navigation!
• Captioning and metadata support, including ISAN.
• Available soon, an integrated publishing module to go straight to Silverlight Streaming.
• Support for a wide variety of source formats, including QuickTime, AVI, Windows Media, VOB (DVD), many other flavors of MPEG-2, MPEG-4, and AVISynth.
• Automation available via both command-line and a .NET object model.
• Simple editing features, like in/out points, preroll/postroll videos, and image and animated overlays.
• GPU acceleration for previews
• A great A/B compare feature, enabling short sections of the file to be encoded with a variety of compression settings, and then compared (playing in real time! zoomed in!, with an A/B split screen slider!) to both the source and each other.
• An unbelievably cool live encoding module, which supports multiple cameras with live switching, streaming from files, including looping, live metadata insertion, and big quality improvements from lookahead rate control and dynamic complexity. I owe you a blog post describing that as well.

So, in summary, I guess I'd say I like it.

I've had a some blog posts highlighting different projects done with Expression Encoder 2 which you can follow along with.

Hands on with high-touch encoding: Streaming Media All-Stars Redo

Encoding screen recordings for Silverlight in VC-1 with Expression Encoder 2

"What Happens in Vegas" - 720p Movie trailer at 2 Mbps via Silverlight Streaming

I've got other information about Expression Encoder and how it fits into Silverlight at

My NAB Presentations

Silverlight Media technologies overview in Expression newsletter

And, of course I'll be covering Expression Encoder 2 as one of the products we cover in my

My June 23-27th class at Stanford