CNET says the ribbon is “the most radical change to the program” in their review, noting that it makes the program instantly recognizable to Office users and helps to highlight the advanced features in a way where users can finally locate them. However, if you’re anti-ribbon, the Classic interface is still available under “Options.”

The new WinZip looks great, but the price ($29.95) is probably a little too high for most of us who simply use the built in compression and extraction tools found in Windows. However, anyone who needs the advanced features WinZip provides should be happy with the upgrade.

• AT&T
• CD Networks
• Internap
• Level 3
• Limelight

• Digital Rapids
• Envivio (and press release)
• Grab Networks (formerly Anystream)
• Inlet (and press release)
• Rhozet
• Telestream (and press release)
• VBrick
• Viewcast (and press release)
• Winnov

Which is a huge swath of the professional compression tools market.

DRM service provider support
And we've had a bunch of support for Silverlight DRM powered by PlayReady from DRM service providers. Since Silverlight encryption is applied during content creation, not during content distribution, using DRM has no real impact on the server side; access to a DRM license server to provide licenses to the Silverlight client is the only big difference from a service perspective.

• BuyDRM (press release)
• Entriq
• ExtendMedia 
• Ipercast

• iStreamPlanet

I haven't had a chance to drop by everyone's booth yet, but will be highlighting some of their demos and cool announcements throughout the week when I find out which of the many projects we've been collaborating on are public now.

Silverlight 3: we're 1080p24
So, we had that whole HD on the web discussion a few weeks ago which spun out into many threads on different forums. A few die-hards said that only 720p60 and 1080p24 should count as full HD, based on the original ATSC definition.

It'll keep on being discussed, but that won't keep anyone from calling Silverlight HD, because we've now got 1080p24 working in Silverlight 3. This represents a huge amount of media playback tuning and testing by the Silverlight team, and it's really paid off. Much of the improvements have been since the public Silverlight 3 beta, so those not at NAB will have to take my word for it for the moment, but we're showing off:

• 1920x1080p24 VC-1 at 6 Mbps
• On a Core 2 Duo 2.4 GHz system
• With smooth playback at any size using GPU scaling

And it is glorious.

On the same system we're also showing off H.264 720p24 2.5 Mbps with similarly smooth playback.

While those specs are obviously beyond what many home PCs can do due to screen size or network speed, with Smooth Streaming we can offer that as the highest-end experience while offering further bitrate bands as low as required.

And of course, optimization that enables 1080p on the high end makes 480p work on lower-end machines than Silverlight 2 could support, due to both media pipeline improvemetnts and offloading scaling and compositing to the GPU. The gains on single-core machines are particularly notable; we've got a quite nice Smooth Streaming experience even on NetBooks now.

The public release of Silverlight 3 will be later this year.

New Smooth-Streaming VC-1 implementation
We're working with the encoder tool vendors to integrate a new VC-1 implementation that's specifically tuned for Smooth Streaming. You may recall Expression Encoder's  Smooth Streaming mode uses the VC-1 Encoder SDK in 1-pass CBR mode with a fixed GOP size. While that certainly can produce good video that's Smooth Streaming compliant, in the end that's really the kind of settings used for live broadcasting. With on-demand content, we can do an analysis pass to figure out a variety of better ways to optimize the bitrate. In particular, we can dramatically reduce the incidence of frames compressed to the point where visible blocky artifacts appear.

What it does are awesome in so many ways that it makes my compression nerd soul twinkle with delight. But those details will have to wait for a long and Excel-chart laden blog post of its own.

In the interim, let me offer you a sample encoded in a not-quite-final version of what we're doing here. This is our favorite Big Buck Bunny content, encoded to these specs

• 1080p24 4 Mbps video with a 5 second buffer
• Smooth Streaming compatible VC-1 and WMA 10 Pro
• In a WMV wrapper, so you can play it in WMP

This is what the top end Smooth Streaming bitrate can look like once Silverlight 3 is released later this year. And as mentioned above, we're now doing 6 Mbps on a Core 2 Duo, we've got some further headroom for more challenging content.

Here's the file (hosted by Silverlight Streaming):
http://silverlight.services.live.com/31260/Big%20Buck%20Bunny%201080p24%204%20Mbps%20Smooth%20Streaming/video.wmv

Remember, Windows Media Player can Save As progressive download content without DRM if you want to make a local copy.

If you're here at NAB, Inlet is showing off the new VC-1 implementation as implemented in their Armada product, with a bunch of other output samples. Check it out.

Silverlight on YouTube

As it’s such a big event, everyone wants in on it. Which has led to what’s the first use of Silverlight on YouTube that I know of. I don’t know why people find this remarkable, but I’ve heard “yeah, but YouTube would never use Silverlight” more the once. So it’s nice to share this screenshot from www.youtube.com/marchmadness.

March Madness Encoding settings

My master compressionist colleague Alex Zambelli and I worked on the March Madness encoding settings and workflow, with him doing all the heavy lifting getting the Inlet Spinnaker encoders configured before the event. He’s included the details in his big Mix news roundup blog post. It’s as worth reading as all his other stuff. I’ll quote the March Madness specific stuff below:

CBS Sports has launched a Silverlight-based March Madness video player that lets you watch all NCAA Basketball Tournament games live. Visit http://mmod.ncaa.com/video to launch the March Madness video player. If you are using Internet Explorer on Windows, the default player will actually be an old-school WMP player, so you’ll need to click on the HQ Player button to launch the new Silverlight player.

The live video for the tournament is being streamed using Windows Media Services. Obviously, we couldn’t use Smooth Streaming because the server technology is still in beta and the encoders aren’t yet commercially available. But CBS did the next best thing! All live streams are available in 4 video quality levels:

Total Bitrate (kbps)	Video Bitrate (kbps)	Audio Bitrate (kbps)	Video Width	Video Height	Pixel Aspect Ratio
1500	1450	48	784	432	1:1
1000	950	48	512	384	4:3
650	615	32	368	272	4:3
350	315	32	240	176	4:3

Video codec used is VC-1 Advanced Profile. Audio codec used is WMA Professional at 44.1 kHz 16-bit stereo.

All March Madness games are being encoded by MLB.com’s encoding facilities using Inlet Spinnaker 7000 encoders. The Spinnakers were configured based on my own recommendations in order to provide maximum quality at all bitrates.

The March Madness Silverlight player uses preroll ad download statistics to estimate available client bandwidth and tries to make an appropriate first choice of bitrate level. Of course none of this would be necessary with Smooth Streaming, but we really tried to make the best of the Windows Media Streaming experience anyway. The player also has built-in heuristics to detect quality-of-service issues, such as frequent rebuffering or low frame rate rendering, at which point it can suggest to the user to choose a lower bitrate. Users can manually switch between available bitrates using the “-” and “+” buttons in the button of the player UI.

Note that we used anamorphic encoding for the lower bitrates. Due to the strong bias towards horizontal motion in basketball, that makes for both more efficient compression (as motion blur is concentrated horizontally, and thus there’s more real detail vertically) and for more efficient encoding, as a smaller motion search range can be used more often. The highest bitrate was encoded as square pixel at the exact size of the MediaElement in the player, so that we are able to use every pixel we have access to. That also mean the highest bitrate doesn’t get scaled, which gives us the Fast Path for better performance.

The IIS team has just released the first public beta of the Smooth Streaming module for IIS7. This means anyone with Windows Server 2008 can start experimenting with this next generation media delivery technology for Silverlight. We’re aiming for a final release in Q2 this year, which will be licensed for full commercial use. There’s lots of detailed info in the links before, but let me offer my own high-level take on what Smooth Streaming is about and why it matters.

Smooth Streaming adapts streaming to the web instead of trying to adapt the web to streaming

Alex Zambelli explored this concept in the links below, but I love it so much I’m going to reiterate. Smooth Streaming leverages the massive scale of the standards-based http web delivery technologies, instead of trying to define different protocols or requiring a lot of bidirectional communications. The media itself is sent as a series of small files as http requests, so it gets automatically cached by the proxy servers in CDNs, ISPs, and corporate firewalls. And of course can pass through firewalls easily.

Smooth Streaming streams smoothly

The core user experience we’re shooting for is quick starting, fast scrubbing, uninterrupted playback of content as good as the local machine can play at the moment. We switch between bitrates and resolutions basic on available CPU speed, network speed, and window size without any “buffering” messages as long as there’s sufficient bandwidth to deliver the lowest bitrate.

Smooth Streaming is standards based

Smooth Streaming uses the http protocol and the MPEG-4 file format (in its “fragmented” flavor). We’re initially using VC-1 (SMPTE standard) and WMA 10 Pro audio, and will support H.264 and AAC-LC support once Silverlight 3 ships with those codecs. The manifests use XML and SMIL.

Smooth Streaming can be authored today

Expression Encoder 2 (with Service Pack 1 installed) includes full support for authoring on-demand Smooth Streaming content, including all media files and manifests. The Silverlight 2 templates included with SP1, and the underlying Base Player, support Smooth Streaming. Other compression tools will be adding full Smooth Streaming support in future versions.

All the Links

We’ve gone blog-crazy here at Microsoft. Which means that there’s lots of great information coming out of us these days, so it can be kind of hard to find it all. Here’s my list of good Smooth Streaming references and tutorials.

Official pages

• IIS Smooth Streaming
• IIS Media

The Demo

• SmoothHD.com (our joint demo site with Akamai)

Downloads

Yep, it’s a free beta download for any version of Windows Server 2008. There are free 60-day trial versions of 2008, including a .vhd download.

• Smooth Streaming Beta for 32-bit Windows Server 2008
• Smooth Streaming Beta for 64-bit Windows Server 2008
• Web Platform Installer (single installer for IIS, SQL Server Express, .NET Framework and Visual Web Developer components)
• Trial versions of Windows Server 2008
• Expression Encoder 2 Service Pack 1 (the full Expression Encoder 2 with SP1 installed is needed to author Smooth Streaming content)

Learn IIS7 articles

• Smooth Streaming for IIS 7.0 - Getting Started
• Smooth Streaming for IIS 7.0 - Exploring Bit Rate Changes
• Smooth Streaming for IIS 7.0 - Managing Your Presentations

My Blog

• Expression Encoder 2 Service Pack 1 – Intro and Multibitrate Encoding (my general overview of Smooth Streaming)
• Encoding Big Buck Bunny for SmoothHD.com (highy nerdy hands-on example… with tables and charts!)

Alex Zambelli

Alex has been doing a great series on multibitrate streaming, adaptive streaming, and Smooth Streaming architecture from a content authoring perspective. They’re great to go through in order

1. A Brief History of Multi-Bitrate Streaming
2. The Birth of Smooth Streaming
3. Smooth Streaming Architecture

John Bocharov

John is an IIS Program Manager who has been intimately involved with Smooth Streaming since its inception.

• Beta Release of Smooth Streaming Now Available
• Smooth Streaming Questions? We’ve Got Answers!
• I’ve seen the future and the future is… Smooth!

I figured this was as good a time as go through a hands-on project for Smooth Streaming.

The Source

We’ve had a few years of good fun with the Blender Foundation’s Elephant’s Dream clip (as I used here, here, and here). They have a new Creative Commons licensed title up called Big Buck Bunny, which is a lot brighter, and with a lot of nice chewy grass for high frequency detail. Elephant’s Dream is a great test clip for handling gradients and dark detail (It’s all about differential quantization), but Big Buck Bunny is arguably a better test for general compression quality, and also has some nice sky gradients as well.

I downloaded the stereo audio as a FLAC file, and converted that to a 44.1 KHz 16-bit WAV file with Audacity. I then downloaded source as a series of PNG files, and then used After Effects to combine the PNG sequence and the WAV that into a 1920x1080 Lagarith RGB AVI file. Since PNG and Lagarith are both lossless, and I didn’t do any scaling or color space correction, I didn’t do any preprocessing, just assembling the content into a file Expression Encoder 2 can consume. I’m a big fan of the quality of EEv2’s Super Sampling scaling mode, so prefer to use that instead of other tools where I have less control over scaling quality.

Setting up the Encode

Since we had other encodes up at SmoothHD.com, I wanted to follow the same data rate/frames size/GOP length parameters we’d already used. For the other encodes in the series. Specifically, the below, which give us seven bands from 2500 Kbps to 364 Kbps:

Tweaking

Smooth Streaming as a feature came in very hot for EEv2 SP1, and there wasn’t a lot of time to tune the presets.

There are a couple of safe tweaks I always recommend as previously mentioned:

• Adaptive Dead Zone=Conservative
• Overlap=On
• Scene Change Detection=On
• Search Range=Adaptive

Those help Smooth Streaming (and web-rate VC-1 in general) quality with minimal impact on encoding time.

For quality-over-speed projects, I also like to use

• Complexity=4
• Chroma Search=Full True Chroma
• Match Method=Adaptive

All together, that’ll increase encode time about 2-3x for a nice but not overwhelming quality difference.

And because this is animation content, I’ll also use

• B-Frame Number=2

In general, the less noise in the content and the closer the matches between frames, the more likely 2 B-Frames will be more efficient than 1. I’ve gone up to 4 B-frames for Camtasia screen recordings.

Charts!

One cool thing about Smooth Streaming encoding is that the independent chunks make analysis and graphing of the content a lot easier. We can analyze each chunk as an independent unit, and compare quality between bitrates and versions quite directly. Do note that there is some allowed variability in chunk size; we target for a 5 second buffer, so that any three chunks in a row should average around the same amount, but if a particular chunk is harder to encode than its neighbors, it can have a somewhat higher bitrate.

Here’s the .XLSX if you want to check out the source data:

I’ve encoded four versions of the content

• Default: the stock EEv2 SP1 Adaptive Streaming settings
• Fast: Adding the “safe tweaks” above
• HQ: Adding the quality-over-speed settings
• Insane: Taking HQ and adding "Insane” modes that add a lot more in encoding time than quality
  • Single-thread encoding
  • Complexity 5
  • Hadamard Motion Match

Insane is there to show what happens when you burn as much CPU as possible.

To show the impact of these different encoding settings on the video, I used an in-house command-line tool to dump a log file of the size, type, etcetera of each frame, and then did made the Excel spreadsheet linked above to see what interesting data I could find.

First off, let’s take a look at just a bunch of frames. Specifically, the frames of Chunk 214, corresponding to the scene where the flying squirrel is flying over the forest floor right before the spikes pop up.

I’ve used contextual shading here to show the quantization parameter for each frame. In VC-1, a 1.0 is the least compressed and 31 is the most compressed. In general, QP is a decent proxy for the quality of a given frame, with values below 8 generally being pretty good and values above 8 getting increasingly blocky.

Since this is a 2-second chunk at 24 frames a second, we have exactly 48 frames. And since it’s a Closed GOP, it starts with an I-frame (keyframe).

The Insane, HQ, and Fast settings offer pretty similar results, due to using the 2 B-Frame pattern and Lookahead to detect scene and other changes. However, QP generally goes down as encode time goes up. The Default encode starts out with much higher QP since the previous scene had used more, and so needed to catch up a bit. By the end of the GOP, its QPs are going down while the “2 B” QPs are going up for a similar reason.

Interesting in itself but it’s definitely a trees-over-forest view.

For 95% of the 10+ minute clip, all the frames look fine; as typical with a CBR encode. It’s in the hard parts where the differences show up.

For the below, I’m going to focus on chunks 209-221 (6:56 to 7:22), which includes the very challenging section where the squirrel is flying over all the spikes jumping up.

In the below, we’re looking at a few metrics here:

• Mean QP per chunk:  The average QP of all frames in the chunk, as a proxy over overall quality.
• Max QP per chunk:  The highest QP in the chunk. Since it’s the ugliest parts of the video that stand out, that’s also an important flag for quality issues.
• Chunk size:  How many bytes in the chunk. Since Smooth Streaming is all about delivering video chunks, chunk size is the primary measure of data rate.

Here’s the data:

You can see there’s quite a bit of variability in chunk size; almost 3:1. That’s fine; we’d expect to see that in content that alternates between easy and hard shots. The swing between Mean and Max QP is worst for the Default settings; Lookhead is handy to help the codec know what’s coming down the pike. For Chunk 217, we see Default has a lower Mean QP but a higher Max QP compared to the others. In general, the Mean and Max QP is highest in the Default, as we can see in the below graphs:

Overall, we see that Default is the weakest, and Insane the strongest. But the gap from Default to Fast is the biggest change; in most chunks, Insane and HQ offer the same Max QP, and HQ has a better mean QP in a few cases. However, there are some chunks where Default does better than any of the other modes; since it’s a 1-pass CBR, one mode may have used more of its buffer by a given point than another, leaving fewer bits for the next part.

For reference, here’s that QP 31 frame in our four versions (click on the thumbnails to open at full resolution).

Default:

Fast:

HQ:

Insane:

So, what went wrong with that particular frame? It was a rate control failure of some sort. We can see from the below that the surrounding frames got more bits and lower QP in Default; it would have been better with a more consistent compression level. It’s also interesting to note the big improvement from Fast to HQ for these sequence of frames; it’s when the content is the most challenging that the slower, higher quality encoding modes make the most significant difference.

When you watch the video at full speed, the QP 31 frame isn’t really that noticeable as it’s right at a scene change.

Given the structure of Smooth Streaming the optimum way to allocate bits is quite a bit different than “classic” web encoding with the VC-1 Encoder SDK used in Expression Encoder was tuned for. Any compression engineer reading this has probably already thought of a dozen ways to improve encoders for chunked video. More about that later.

Lastly, here’s how long each version took to encode.

• Default: 59 minutes
• Fast: 59 minutes (1.0x Default)
• HQ: 158 minutes (2.68x Default)
• Insane: 268 minutes (5.4x Default)

As you can see, you rapidly get to the point of diminishing returns with slower encoding modes; the biggest improvement was from Default to Fast, which had no impact on encoding time at all.

Is VC-1 the same as Windows Media Video 9?

Yes, the big picture: VC-1 is the SMPTE standardized version (SMPTE 421M) of Windows Media Video 9. The output of of a VC-1 encoder is bitstream compatible with a Windows Media Video 9 decoder that supports the profile, level, and features used.

Is VC-1 the same thing as WMV?

Note that the VC-1 spec just defines the video bitstream; it doesn’t cover the ASF (used in .wmv and .wma) file format or any WMA codec. VC-1 Advanced Profile can live in a variety of other file format wrappers, including MPEG-4 (standard or fragmented, the latter being used in Smooth Streaming) and MPEG-2 transport stream (as used on Blu-ray or IPTV). Only VC-1 Advanced Profile has this kind of format independence; VC-1 Simple and Main profiles are always found inside a WMV file.

Can I use Expression Encoder to make files that play in Windows Media Player 9?

Yes, absolutely. VC-1 Main Profile is identical to the old Windows Media Video 9. A Windows Media file you make using VC-1 Simple and Main will work perfectly in WMP 9 out of the box. A stock XP Service Pack 2 machine that hasn’t been updated in years would play it without issue.

Will users have to install a update if I encode with VC-1?

No, everyone with Windows Media Player 9 or higher can play VC-1 Main Profile already. Windows Media Player 11 can play Advanced Profile as well. Users or WMP 9 or 10 will be prompted to install a VC-1 Advanced Profile decoder if they try to play a WMV file using that.

Can I encode for Blu-ray from Windows Media Encoder?

So, Since Windows Media Video 9 Advanced Profile is the same as VC-1 Advanced Profile, and VC-1 AP is supported on Blu-ray, can you encode a Blu-ray compatible stream in Windows Media Encoder?

Nope, sorry. There are further constraints for Blu-ray VC-1 that need to be followed, and WME and the Windows Media Format SDK weren’t designed to support that level of constraint. The VC-1 Encoder SDK was designed with optical discs in mind, and can produce a Blu-ray compatible VC-1 elementary stream with products that expose it. Expression Encoder 2 doesn’t do elementary streams, but Rhozet’s Carbon Coder  3.12 and higher do.

How do Windows Media Video 9 and VC-1 Profiles relate?

Here’s the details.

Following up on the above, there’s a couple of points I’d like to make clear

• Windows Media Video 9 originally had three profiles: Simple, Main, and Complex. Simple and Main are the same as the VC-1 Simple and Main Profiles. Complex was deprecated, and evolved into Advanced Profile. As Complex isn’t part of the VC-1 spec, it will normally only work on decoders from Microsoft, or based on Microsoft’s porting kit implementations.
• There were two flavors of Windows Media Video 9 Advanced Profile. The first, with a “WMVA” 4CC, was introduced with WMP 10 and used as the basis for VC-1 but some changes were made in the final specification that kept it from being entirely compatible. An updated VC-1 compliant version of Windows Media Video 9 Advanced Profile with a “WVC1” 4CC was introduced with WMP 11, and that is what should be used.
• Anyone wanting to make WMV 9 Advanced Profile needs to either have Format SDK 11 installed (the easiest way is to install WMP 11) or use a VC-1 Encoder SDK based product like Expression Encoder 2.

This is where Simple/Main/Complex profiles are set in Windows Media Encoder:

If it’s the same codec, why are there different Windows Media Video 9 and VC-1 Encoders?

All that said, there is one last practical difference between WMV 9 and VC-1: Encoders that call it VC-1 are normally based on the newer, faster, higher quality, more flexible VC-1 Encoder SDK. This isn’t about a change in the codec standard, just the usual improvement of encoder implementations over time, like we saw with the Format SDK 9, 9,5, and 11 releases, and will be seeing again in future releases.

Here’s a comparison of the gains from the initial release (1st generation) to VC-1 Encoder SDK (4th generation).

Further Reference

• http://alexzambelli.com/blog/wmv9-vc1-faq/
• http://alexzambelli.com/blog/vc-1-sdk-faq/
• http://www.microsoft.com/windows/windowsmedia/forpros/codecs/video.aspx#WindowsMediaVideo9AdvancedProfile

As I mentioned on Friday, the official streaming feed of Barak Obama’s Presidential Inauguration Committee will be using Silverlight 2.

However, using managed code in the player leaves out Linux users of the Moonlight 1.0 beta, as well as PowerPC Mac users. We’ve heard a lot of requests to add support for those platforms, and so we teamed up with the Moonlight team at Novell and they’ve created a Silverlight 1.0 version of the player that works great in both Moonlight and PowerPC Macs. Miguel de Icaza has details from his side.

We kicked the project off this afternoon before the event, so pardon us if it’s a little rough. The only significant limitation is that the SL1 player can only do the 500 Kbps stream – lacking managed code, making a manifest XML parser in JavaScript didn’t seem like something we should start testing only nine hours before the event goes live.

But anyway, thanks to Joseph, Larry, Geoff, Rusty and especially Aaron at Novell for pulling this together so quickly and so well, and Mio at iStreamPlanet for making the last minute encoder tweaks to make it Silverlight 1.0 compatible. On the Microsoft side, Brian Goldfarb and Eric Schmidt did a great job making the call and making it happen. Just go to www.pic2009.org in the morning and the updated player should be available around 6 am EST (11 am GMT).

That something like this could come together on such short notice is a testament to the chops and passion of the Moonlight team and the great platform we’re both implementing. I’m glad we got this chance to demonstrate how serious we are about this collaboration (although I wouldn’t have minded starting a day earlier…).

Update: The Silverlight 1.0 compatible player is low live at:

http://www.pic2009.org/page/content/linuxplayer

Note that this player is only for Silverlight 1.0 and doesn't work with Internet Explorer. The Silverlight 2.0 player for Windows or Intel Mac is at:

http://www.pic2009.org/page/content/live

(And yes, that would have been easy to fix, but we're rather out of time!)

Tech Specs

And as I promised earlier, I got the tech details on how the streams are encoded. Note there is manual stream selection in the lower right corner;

Hardware

There are quite a few encoders to handle the different streams, data rates, and to provide failover backups. All systems are quad-core, and use hardware preprocessing.

• Onsite configuration: Dell Precision workstations with Osprey 230 capture cards
• Offsite configuration: Dell 2950 with Digital Rapids capture cards.

Software

The encoders are running Windows XP and use Windows Media Encoder, with my recommended tweaks and registry key settings applied.

Codecs

• Video: Window Media Video 9 Advanced Profile (aka VC-1 Advanced Profile)
• Audio: Windows Media Audio 9.2

The audio was originally going to be WMA 10 Pro for improved efficiency, but we fell back to WMA 9.2 in order to have Silverlight 1.0 compatibility. Fortunately the audio feeds are either mono or might as well be, so we can win some efficiency back by encoding in mono.

300 Kbps streams

• Video: 480x360, 29.97 fps, 259 Kbps
• Audio: 44.1 KHz mono, 32 Kbps

500 Kbps streams

• Video: 480x360, 29.97 fps, 442 Kbps
• Audio: 44.1 KHz mono, 48 Kbps

First, install it already! It can be downloaded here, and probably most of you picked it up via Microsoft Update by now. James Clarke had a great overview blog post about it which I recommend.

This first post I’m going to focus on Smooth Streaming and multibitrate streaming in general.

“Adaptive Streaming” and Smooth Streaming

Here’s the big one: Expression Encoder can now do multibitrate encoding. Alex Zambelli has a good history of the technology.

When doing multibitrate encoding, EE does multiple simultaneous encodes from the same source file. While the big driver for that feature is Smooth Streaming (another overdue blog topic), we also support encoding a single multiplexed “Intelligent Steaming” WMV file or individual WMV files for each bitrate.

SmoothHD.com

And If you haven’t seen it yet, head on over to the SmoothHD.com demo site we do with Akamai’s  and check out the technology in practice. And yes, all those files were encoded with SP1.

SmoothHD.com in action. Those bars in the lower right show which data rate band you’re getting; mouse over for more details. Click on it to bring up a diagnostic menu and play around with scaling. And boy, that 20 Mbps DSL upgrade I’ve got on order can’t come soon enough!

Previewing Smooth Streaming with Expression Encoder

So, you want to play around with Smooth Streaming, but don’t want to have to upload your files to Akamai every time you want to do a quality check? Fair enough, but we don’t have media players that support the Fragmented MPEG-4 format used in .ismv files yet. Fortunately, EEv2 SP1 also includes a little localhost Smooth Streaming web server built into it. If you encode to a Silverlight 2 template and make sure that “Preview in Browser” is checked, you’ll get Smooth Streaming working, heuristics and all! It’s only on the local machine, and only when EE is running, but it’s a start.

To preview the video quality of Smooth Streaming, you can also make individual WMV files and watch them as normal. The output bitstreams are identical no matter what the file is wrapped in.

WMSnoop remains a great (and free) way to check out the GOP structure of WMV files. Below (click to zoom) you can see how Smooth Streaming produces a rock-solid cadence of a keyframe (in yellow) every 60 frames, while still being able to insert extra keyframes as needed for high quality scene changes.

Encoding for Multibitrate

Unlike the old FSDK-based MBR encoder, each stream is encoded in its own thread, which means a whole lot of cores can be saturated; my 8-core typically is running at nearly 100%, and if HD is involved, a 16-core would as well. The limiting factor can wind up being the speed of the source decoder past a certain point. Here’s a pretty typical load I’ll see when running from a Lagarith source file.

Multibitrate Video Options

Most of the parameters must be identical per stream; only frame size and bitrate can vary each. Other parameters like frame rate, audio, and codec settings are fixed. This is to facilitate Smooth Streaming by making sure that I-frames appear at the same point in each output stream. For technical reasons with the current VC-1 SDK, this also means only 1-pass CBR is supported for multibitrate encoding.

Multibitrate Audio Options

Only one audio option can be selected when doing Adaptive Streaming encoding. This isn’t the limitation it would have been in the WMA days, as WMA 10 Pro’s low bitrate audio quality is so good. We find 64 Kbps provides great quality for most content, and fits into most web delivery bitrates.

Multibitrate Container Options

There are three container choices

• ASF Single File. This gives you one WMV file for each video data rate, each including its own copy of the audio. This is an easy way to make multiple versions of the same source file.
• ASF Multiple File. This gives a single WMV file containing all copies of the video and one copy of the audio, ala Intelligent Streaming. This provides bitrate switching when used on Windows Media Services.
• IIS Smooth Streaming. This is for use with the forthcoming IIS Media Pack version that enables Smooth Streaming. To use this today, you’ll need to be part of Akamai’s beta program. That gives one output file per stream. There’s no way at the moment to play these files easily on the desktop, as they use the Fragmented MPEG-4 file format, which isn’t broadly supported in media player software; it’s mainly used inside of set top boxes like a TiVo.

Multibitrate Advanced Codec Settings

There’s a few required settings to get optimal Smooth Streaming compatible content in EEv2SP1. Smooth Streaming requires that all bitrates start have Closed GOPs (Group of Pictures – a keyframe and all the frames that reference it) starting on the same frame every few (typically 2) seconds. This means that the decoder gets a continuous sequence of frames to decode without any overlaps or missing frames, and without having to run two simultaneous decoders. These are compatible with (but not needed for) ASF, and using them gives the option to remux to Smooth Streaming at a future date.

• Maximum QP=31. This is a new feature in SP1, which specifies the maximum allowed Quantization Parameter that can be used. This is similar to the old WME “Smoothness” control which determines the tradeoff between preserving image quality versus frame rate. We’re trying to avoid a frame getting dropped in one data rate but not another, which could throw off GOP alignment. QP 31 is the most compressed a frame can be in VC-1, thus minimizing the chance of a skipped frame.
• Adaptive GOP=Off. Adaptive GOP tells the codec that it can “reset” the Key Frame Interval, making it into really “keyframe at least every.” That improves efficiency, but makes it possible that the different output bitrates might wind up starting new GOPs at different times, without any mechanism to resync them. When Adaptive GOP=Off, it’ll still insert an I-frame where appropriate to optimize quality, mainly when there’s a video cut in the middle of a GOP.
• Closed GOP=On. In a Closed GOP, the first frame of the GOP is always an I-frame (keyframe), and no frame in the GOP references any other GOP. That makes each GOP fully self-contained, as required for Smooth Streaming. If Closed GOP were off (Open GOP), B-frames can reference frames in other GOPs. This helps efficiency a little bit, but means you always need to have decoded most of the previous GOP to play the first frame of the current one. Even if Closed GOP=On, any additional I-frames inserted due to scene changes may be an Open GOP for improved efficiency.
• Output Mode: Elementary Stream Sequence Header. With an Elementary Stream Sequence Header, the VC-1 bitstream includes all the data the decoder needs to play back the chunk of video in that chunk. It’s required (and it’s a tiny bit of data; there’s no downside to using it).
• Insert Skipped Frames=On. With Skipped Frames, a frame doesn’t get encoded due to exceeding the Max QP or because it’s just a static frame without motion, consist of just a flag indicating “I’m the same as the frame before.” If Skipped Frames=Off, the frame before the dropped frame has its duration extended to cover the missing frame(s). The actual video playback is identical in either case; but this  ensures dropped frames don’t cause a GOP misalignment. There’s no real downside to using this for all files, Smooth Streaming or not.

Tips for multibitrate encoding

General Codec tweaks

These are the changes I made from the Adaptive Streaming default, implementing the typical “make it look good” settings I recommend for EE in general also apply to multibitrate compression. Just don’t mess with the settings up above!

Always On recommendations

These settings won’t have much impact on encode time, but can help quality significantly, particularly with

• Adaptive Dead Zone=Conservative . This tells the codec to bias toward softness over blockiness as a frame gets more compressed.
• Overlap=On. The Overlap filter softens the edges of blocks, reducing blocking and artifacts at higher compression rates. Web delivery is almost always at rates where the Overlap filter helps quality.
• Scene Change Detection=On. Scene Change Detection tells the codec to buffer ahead 16 frames to look for fades, flashes, and scene changes, and dynamically change the frame type based on that. This particularly helps with fades to/from black, and with flash frames (think raves and gunfights). While it can insert an I-frame, it won’t disrupt the pattern locked in by having Adaptive GOP=Off. Scene Change Detection never hurts quality, and often helps. Scene Change Detection maps to the Lookahead parameter used in the Format SDK.
• Search Range=Adaptive. Lets the codec dynamically adjust the search range based on the amount of motion in the content. Helps quality a lot when there’s more than 64 pixels horizontally or 32 pixels vertically of motion between two P-frames, without spending CPU cycles on frames with more sedate motion.

Quality over Speed recommendations

These are settings I use when I’m more worried about optimum quality than the speed of encode.

• Complexity=4. Complexity controls how hard the codec works on each frame, particularly the precision and thoroughness of looking at motion from frame to frame. Complexity 3 is a great default, since it provides most of the value of 4 and 5 at a higher speed; quality drops of quickly as you go to 2 on down. But 4 is a little better yet, particularly with complex motion like particle effects. Complexity 5 is overkill most of the time, but I wind up using it sometimes when I’m feeling more fussy than rushed, “just in case.”
• Chroma Search=Full True Chroma. Chroma Search looks for where the chroma (color) of the image changes differently that its luma (brightness).. This is particularly helpful with motion graphics and animation. Chroma is only a third of the data in the encode, so a full precision chroma search doesn’t add all that much to the encoding time, and with the right content can really help squeeze things down.
• Match Method: Adaptive. This switches between using Sum of Absolute Differences (SAD) and Hadamard motion matching on a per-macroblock basis. Don’t sweat what that means too much; just know it’s slower and a little faster than the default SAD-only. Full Hadamard is slower yet and sometimes lower quality than Adaptive.

Stretch, not Letterbox

Expression Encoder defaults to letterboxing to be used when the output frame size doesn’t match the source aspect ratio. However, even a slight variation in output aspect ratio from the source’s (typically to get the encoded height and width divisible by 16, the most efficient option) can result in a thin black line of letterboxing (top and bottom) or pillarboxing (left and right). If Resize Mode=Stretch and Mode=Profile Adaptive, Profile Adaptive will get the output frame size in the right ballpark while Stretch adjusts the video slightly to avoid those darn black lines. To make sure the output aspect ratio really is perfect, you can set the Video Aspect to that of the source, and on playback it’ll stretch the image pack to the exact original frame size and shape. Leaving it at the default of Square Pixel generally looks fine as well.

Note that you may need to reapply the Profile Adaptive setting if you apply a job preset to multiple files at once. That can reset to Mode to Custom.

Integer Framerates

By default, Adaptive Streaming profiles gives a keyframe every 2 seconds. However, that’s a precise 2.0000 seconds. So, if your source is a typical NTSC frame rate like 29.97 or 23.976, you’re 0.1% short of 30 and 24 frames a second, and so wind up getting a keyframe every 47 or 59 frames, not the 48 or 60 you might expect. If you set the frame rate to the whole number equivalent, the encode will actually maintain the source frame rate correctly (the value serves as an upper bound, but EE won’t insert duplicate frames when set higher than the source). But it will round up the keyframe cadence to the whole number, so you’ll get the keyframe every 48 or 60 frames like you’d expect.

Not really a big deal in practice I suppose, but I’ve long since internalized the multiples of 48 and 60, which makes it easier to me to figure out where my keyframes are.

Determining bitrates

Picking the optimum single frame size and data rate combination for a particular piece of content can already be a challenge. Having to determine optimal sizes for a bunch of different data rates. Fortunately, Alex Zambelli is as always ready with a handy utility for us – the Smooth Streaming Multi-Bitrate Calculator!

All you need to know is

• What bitrate works for the highest frame size for your content
• What minimum bitrate you want to go to
• How many steps to take

There’s actually some pretty deep math in there that knows how VC-1 bit-per-pixel requirements change as frame size changes, etcetera.

And my compression classes are approaching apace, with signups for Stanford already open, and Portland State just a few weeks away! Beyond all the changes in the technology, I’ve got a structural change as well: the Portland State class is now going to be on three consecutive Saturdays, making it easier for people in the region to take without taking time off from work.

These classes are among the highlights of my year. I never learn faster than when I’m teaching, particularly when I get the great students that attend the sessions. Skill levels vary widely, and the course is designed to accommodate that. But everyone’s got something unique they’re trying to do, whether it’s a supervisor of a high-volume compression department getting up to speed on new formats, or an educator incorporating videos of marine animals into the classroom. And it’s hose real-world projects where the rubber meets the road. The focus of the classes is on hands-on art, science, and craft of video compression. It’s all about to get the best results out of real world content with real world workflows, within all the real-world constraints we have to operate under.

When Microsoft was recruiting me back in 2005, one of my top requirements was that keep on teaching these classes, with full freedom to cover the formats and technologies that matter, even if competitive with our own. It was an easy sell – they understand the value of me understanding everything. And of course, now that VC-1 is a SMPTE standard and Silverlight is getting H.264 support, the era of proprietary media formats is over anyway. So while we’ll certainly spend time with VC-1, WMV, and Silverlight, we’ll also cover MPEG-2, MPEG-4, Flash, DVD, Blu-ray, Ogg Theora, and other formats and players based on class interest.

Class time is roughly split between lectures/demos and hands on time doing projects. Each student gets their own workstation loaded with the latest and greatest compression software and related tools.

And I really encourage students to bring along some of their own content and projects, particularly one’s they’ve been having trouble with. Nothing beats that kind of variety of real projects to teach the tips and tricks of our craft.

Note that both classes offer academic credit, and so are generally covered by corporate education benefits.

Portland State University: Feb 7, 14, 21 (10am-6pm)

Compression for Digital Media (MP404)

This class is part of the PSU’s Multimedia Professional Program, one of the first digital media degree programs in the country. The class is available as an elective to those not in the program as well (we’re perhaps 50/50).

Since it’s on Saturdays, I expect we’ll see more commuters from around the Pacific Northwest this time around.

And it’s a rare pleasure for me to have something work-related to do that’s a 10 minute drive instead of an hour flight!

Stanford University: August 10-14

Mastering Video Compression

This is the one that started it all; 2009 makes it a full decade since the very first 2-day class I did for the Stanford library science department on authoring QuickTime for education . We’ve been doing the current week-long format for eight years now. The program that ran that class evolved in the Digital Media Academy, which now runs a very wide variety of classes. My 9 year old son came along last year to take a great LEGO Robotics course the same week. He and James Clarke (who took the class) really hit it off; the three of us can deliver quite a whirlwind of nerdish intensity.

Since it’s a one week intensive, it works as a destination class; we get people flying in from around the world. On-campus dorm rooms are available (and quite nice; I stay in one), with other lodging options available, and a meal plan.

Anway, that’s the schedule. If anyone has any questions, feel free to ask below or contact me directly.

To whet the appetites of my compressionist friends, one much-requested new feature in the Format SDK .dll in Windows 7: It now can go up to 8-way threading, compared to the 4-way threading introduced with WMP 11 and Vista! This should nearly double encoding speed on 8-core machines doing HD encoding.

The number of threads the encode gets is determined by the height of the video frame.

• 1-119 lines 1 thread
• 120-239 lines: 2 threads
• 240-479 lines: 4 threads
• 480+ lines:  8 threads

This is because each thread gets a slice of the video frame, so more threads mean that vertical motion gets a little less efficient to encode. And as always, the number of threads used can be overrridden with Alex Zambelli’s WMV9 PowerToy. Its looks like we’ll get one last update out of him after all to add “8” to the drop-down list.

This new mode will work with any program that uses the Format SDK .dll, including Windows Media Encoder, the new Windows Live Movie Maker beta, and Virtual Earth. It won’t apply to VC-1 Encoder SDK based apps like Expression Encoder 2, but obviously we’ll want to add this to the VC-1 Encoder SDK as well.

So, if you’ve got an 8-core processor, or a 4-core with hyperthreading, give the Win 7 beta shot. I’d love to hear from anyone who gets some comparisons for encoding time between XP/Vista and Win 7.

A fellow called Kurtnoise13 jumped on it on the Doom9 Forums discussion about the SDK. He quickly created a new version that supports AVISynth files as source as well. The syntax for a simple PAL 1.2 Mbps encode (from the ReadMe) looks like:

AVS2ASF -i VideoWithAudio.avs -o file.asf -framerate 25 -rate 1200 -profiletype 2 -bframes 2 -maxkeydist 6 -vbv 300000 -peakrate 2000 -pr AudioEncode.prx

Be warned that while the syntax looks a lot like WMCmd.vbs the details can be deceptively different. In particular, units can be different (Kbps instead of bps makes a big difference!). Also, since the VC-1 Encoder SDK is video only, a PRX file defining the audio setting also has to be used; the Format SDK is called to encode audio and mux to WMV.

AS2ASF also supports elementary stream output, so it’s able to generate a .vc1 file for Blu-ray authoring.

For those not familiar with it, AVISynth is an open source frameserver that dynamically processes video and lets it look like any DirectShow source to other tools. This means that intermediate files aren’t needed – an AVISynth “.avs” script can define the source and all the processing to apply to it. And there’s a huge community of AVS plugins to handle new format times and processing modes; almost any kind of file out there can be handled by AVISynth, and almost any kind of processing you’d like to apply is available. For a simple example, the below script

• Opens a PNG sequence at 24 fps
• Associates a WAV file with it
• Crops it from 640x360 to 640x352
• Outputs it in YV12 (8-bit 4:2:0)

a=imageSource("D:\Elephant's Dream\PNG 360\%05d.png",1,15691,24)
b=wavsource("D:\Elephant's Dream\ED-CM-St-16bit.wav")
audiodub(a,b)
crop (0,4,0,-4) # crop height 352
ConvertToYV12()

Anyway, I wanted to highlight this technology. The source code is available as a Visual Studio project, and I’d love to see other members of the community continue to enhance it.

A few things that’d be cool:

• A simple batch automation GUI
• A simple automatic AVS generator, so a .avi or .mov file could be specified as source and the right AVS to wrap it could be build on the fly.

Bear in mind that Windows Media Encoder is older than five years now, which is when multimedia products start moving from Mainstream Support to Extended Support in the support lifecycle. We don’t set support policy for unreleased products, so note that there’s been no formal guidence on whether WME will be supported on Windows 7.

Hotfixes

WME is ancient code at this point, and all updates since the original Windows Media 9 Series launch have been done as hotfixes. You need to have these installed for security, stability, and performance.

Vista compatibility hotfix

If you’re on Vista, you’ll absolutely want to install this one.

FIX: You may experience issues when you use Windows Media Encoder 9 Series on a computer that is running Windows Vista

I covered this in my first non HD DVD blog post here on On10.net!

It can be a bit tricky to install sometimes, so follow these instructions from the invaluable

Windows Media and DirectShow tips, gotchas and code bits blog.

This hotfix address three issues:

Issue 1

When you run the Windows Media Encoder command-line script WMCmd.vbs, the script host Cscript.exe may crash.

Issue 2

The icons that appear on the encoder toolbar and in the encoder dialog boxes are displayed by using a low bit depth. Therefore, the icons appear to have a low resolution.

Issue 3

When you configure the encoding profile or start an encoding session, the encoder may crash.

Multithreaded Multiple bitrate encoding hotfix

This one fixes a pretty embarrassing bug – Format SDK 11 wouldn’t use multiple processors when encoding in multiple bitrates (aka “Intelligent Streaming”). Originally blogged here.

FIX: The WMV9 encoder multi-bitrate performance on a multiprocessor system is severely impaired

Critical Security Fix

This is a security fix for a critical vulnerability that could allow remote code execution on a machine running WME. Install it before you launch WME! Note that the vulnerability requires two things that shouldn’t be happening anyway on a production encoder

• WME is running logged in as an administrator
• The machine is being used to browse the web

Microsoft Security Bulletin MS08-053 – Critical. Vulnerability in Windows Media Encoder 9 Could Allow Remote Code Execution (954156)

Current Format SDK version

WME in the end is mainly a graphical front-end to the Windows Media Format SDK. The actual codec version you’re running is determined by the SDK. Most people get those updates bundled with Windows Media Player, so as long as you have the most recent WMP version for your platform, you’re good to go.

If there’s some reason that you don’t want to update WMP but still want the current codec .dlls, you can also install the Format SDK directly. The SDK version also includes lots of samples and all the normal SDK goodness for building apps for authoring Windows Media content.

For vendors of compression tools, it’s always a good idea to chain the redistributable installer (WMFDist11-WindowsXP-X86-ENU.exe) along with your installer, so you can ensure that users have the current version. That installer can also un manually to install .dll updates without updating .

I’ve seen way too many production encoders in the last few months that have WMP 9 and thus the 9.0 SDK. That’s a good six years old now, and will offer much lower quality and perf than the current versions deliver. Remember this comparison?

Windows XP, Vista, or Server 2008: Format SDK 11

Windows XP shipped with Windows Media Player XP, which was a flavor of WMP 7. However, the almost universally installed Service Pack 2 included WMP 9. WMP 11 is also available for XP, and should absolutely be installed on any XP-based encoder (and really all XP machines in general).

Windows Vista shipped with WMP preinstalled, so no action is required there.

On Windows Server 2008, Windows Media Player, and hence the FSDK, is installed as part of the Desktop Experience feature. You’ll want to enable the DEP on any encoder box (and in general for any 2K8 box you’ll be using as a workstation).

Windows Server 2003: Format SDK 9.5

The most recent version for Server 2003 is the older Format SDK 9.5 (which is still better than 9.0). As service to our customers looking for a high degree of confusion, Format SDK 9.5 was released with Windows Media Player 10.

Note that running on Server 2003 means you won’t have access to the SMPTE-compliant “WVC1” flavor of Windows Media Video 9 Advanced Profile, nor the excellent WMA 10 Pro LBR audio codec, and hence the 32-96 Kbps range for WMA Pro. The codec is also only 2-way threaded instead of 4-way threaded, slower in general, and lacks support for Tarari acceleration and the advanced registry keys.

If you need to encode on a Server OS, you should use a VC-1 Encoder SDK based product or upgrade to Server 2008 (ideally both!), which will give you better video. However, WMA 10 Pro will still not be available on Server 2003.

There have been reports of people using the WMFDist11-WindowsXP-X86-ENU.exe installer from FSDK 11 set to XP compatibility mode to install the FSDK 11 .dll files onto Server 2003. While there aren’t known issues with this, I should point out that this isn’t a supported configuration for Server 2003, and so Microsoft support will not be available for any issues when running with mismatched .dll files.

64-bit Windows: Format SDK 11 (32-bit)

While there is a “Windows Media Encoder 64-bit edition”, don’t use it (I’m not even going to link to it). It predates FSDK 11, and offers lower quality and performance than FSDK 11 does; the performance advantage of running 64-bit native code is a lot smaller than the other improvements in FSDK 11.

So, keep using the 32-bit stuff on 64-bit as well as 32-bit versions of Windows.

Registry keys

Once upon a time, this blog seemed to be mainly about using special registry key options in the Format SDK. Thankfully, with Expression Encoder and the other VC-1 Encoder SDK based products mean we have GUI access to all of these now in modern products.

First off, install Alex Zambelli’sWMV9 PowerToy, which is a simple .NET GUI for setting the codec settings. The tooltips are a better reference than anything I’ll type here; I’ll just give some basic recommendations for different scenarios.

“Always On” choices

This is a good set of defaults with no significant performance downside that almost always helps quality.

Perceptional Option: Adaptive Dead-Zone 1. This maps to the “Adaptive Dead Zone: Conservative” option from Expression Encoder. This lets the codec reduce detail before introducing artifacts, and generally improves quality at lower bitrates.

In-Loop Filter: On. This turns on the In-Loop deblocking filter which softens the edges of block artifacts. This improves the current frame, and also future frames based on it.

Overlap Filter: On. This further smoothes the edges of blocks. It can reduce detail a little at high bitrates, but is almost always helpful at typical web bitrates.

B-Frame Number: 1. Turns on B-frames, and hence enables flash/fade detection when using Lookahead or 2-pass encoding, and also improves compression efficiency.

Lookahead: 16. Tells the codec to buffer ahead 16 frames in 1-pass (CBR or VBR) encoding, letting the codec detect flash frames and fades and switch the frame type based on it. Maps to the “Scene Change Detection” option in Expression Encoder. It will increase end-to-end latency by that many frames in live encoding, but is generally worth it due to quality improvements.

High Quality Live

Assuming you have a fast enough machine to run these settings in at least Complexity 3, they will improve the live experience.

Motion Search Level: Fixed Integer Chroma. This adds basic chroma search to the encoding, which can help the quality of motion graphics significantly. It’s a pretty small boost with more typical camera-shot content, so turn it off if you have perf issues; getting to Complexity 3 is more important.

Motion Search Range: Adaptive. This tells the encoder to switch to a bigger motion search range for frames with high motion, and then go back to a smaller range when motion dies down. This dramatically improves quality with higher motion at bigger frame sizes. The default range is 64 pixels left/right and 32 pixels up/down, so if any objects move more than that between any two P-frames (if you’re using B-frames, that’ll be 2 frames apart), than this feature will help.

High Quality Offline

These settings offer maximum quality for offline encoding, and are slower yet. Use them when you’ve got the time, and run at Complexity 4. Complexity 5 ignores the Motion Search Level and Motion Match Method settings, making it lower quality. The only time you need to use Complexity 5 is if you can’t set registry keys (in which case it’s quite a bit better) or need to use WME to scale your video, as scaling quality is much better in Complexity 5. However, you’re better off preprocessing in another tool if at all possible.

Motion Search Level: Fixed True Chroma. This is a full-precision motion search for chroma. It never hurts, and can help quality a lot with motion graphics and animation.

Motion Match Method: Adaptive. This switches between the Sum of Absolute Differences (SAD) and the Hadamard method to compare motion between frames as appropriate for each macroblock. Full Hadamard can be higher quality for some very complex content, but the Adaptive mode is faster and better most of the time.

Use hardware preprocessing

WME’s preprocessing was designed for good live encoding performance on circa 2002 machines, which means it’s tuned far more for speed than quality. In particular, deinterlacing and scaling aren’t very good (to say the least). So if you’re using WME with a hardware capture card like an Osprey, you’ll get better results doing all the preprocessing on the card, and just passing the final scaled, cropped, deinterlaced YV12 bitmaps off to WME. This also saves some additional CPU cycles for the codec. For file-to-file encoding, you also want to do any deinterlacing and scaling before you import the file into WME. Expression Encoder has great scaling and good deinterlacing, so that’s another reason to use it over WME.

Optimum hardware

If WMP 9 is all you have on your encoder because that’s all there was when you configured it, you’re going to be running on some very old and very slow hardware! Bear in mind that Moore’s Law predicts you can get twice the computer for your dollar every 18 months or so. So today’s best machines will have about 16x the encoding horsepower of what you could have had when WME was released!

Encoder Complexity

When tuning hardware, the goal is to enable higher encoder complexity values to be used. Your target is at least complexity of 3; that’s most of the way towards optimal quality; lower values sacrifice a quite a bit of quality for improved speed, while higher values sacrifice a lot of speed for only a little additional quality improvement. Complexity 4 is optimum when time/performance isn’t a concern. As mentioned above, Complexity 5 in FSDK 11 invalidates some registry keys (not an issue with VC-1 Encoder SDK).

For live encoding, the best way to test is to run the encode using real-world sources and make sure you’re not dropping frames at your target frame size. You’ll want to set Video Smoothness to 0 for this test to make sure frames aren’t being dropped for quality reasons instead of performance reasons.

CPU

Since the encoder is 4-way threaded, you want to be on at least a 4-core machine. These are dirt cheap these days, with affordable 4-core laptops coming soon. Generally speaking, a 4-core is a lot faster than a 2-core at a slightly faster clock speed.

Clock speed is a big factor as well, but it only matters in context to the generation of processors. For example, the new Intel i7 “Nehalem” get a whole lot more work done at 3 GHz than the original P4 “NetBurst” CPUs did at 3 GHz.

Memory

Memory isn’t a big factor in encoding, as long as you have enough. You never want an encoder to start swapping, as performance will fall through the floor. 2 GB should be plenty for a dedicated encoder, even doing HD encoding.

Tarari Encoder Accelerator

If you’re encoding to higher resolutions and just can’t hit the performance you need at a sufficient quality, check out the Tarari Encoder Accelerator. This offloads a number of the computationally intensive parts of compression to a PCI-X board. While it can’t produce the absolutely best quality a tweaked Complexity 4 + registry key software encode can, it can certainly deliver much higher quality for live HD than pure software, particularly on older computers.

The Tarari board is also fully supported by VC-1 Encoder SDK products, so your investment isn’t tied to a FSDK workflow.

Move to a VC-1 Encoder SDK based product!

So, that’s how to get best results out of legacy Format SDK encoders, particularly WME. But really, it’s time to start planning a migration to a VC-1 Encoder SDK based encoder. If there’s something you need that they simply don’t provide, please let me know. Future improvements in VC-1 and Windows Media encoding are going to focus on the SDK, so the gap between Windows Media Encoder and the best encoders is going to be growing bigger over time.

Extra – rollup of Windows Media Encoder related posts

And just so they don’t disappear off the bottom of the blog, here are my blog posts about WME and the FSDK 11 codecs that readers may still find useful:

Low Latency webcasting with Windows Media and Silverlight

Compression for Silverlight Webcast posted

New WMcmd.vbs with improved multithreading and new presets

http://on10.net/blogs/benwagg/New-PowerToy-and-registry-key-documentation/

Hands-on with the Temporary Insanity trailer

http://on10.net/blogs/benwagg/Encoding-for-the-Zune/

http://on10.net/blogs/benwagg/Elephants-Dream-720p--2-Mbps/

http://on10.net/blogs/benwagg/My-Live-Webcasting-with-Windows-Media-presentation-from-Streaming-Media-East/

http://on10.net/blogs/benwagg/My-Mix-07-session-about-WMV-Encoding-and-Expression-Media-Encoder/

It’s full of awesomeness, but I wanted to excerpt the section describing their 1st gen, 2nd gen, and HD encoding settings and workflow.

First Generation Encoding

Our first set of encodes are based on WMV3 and WMA in ASF with WMDRM10 (Janus). We chose these standards because the Janus components have been widely adopted by our CE partners such as Roku, LG Electronics, Samsung, TiVo, and of course Microsoft Xbox.
We encode most content at 500, 1000, 1600, and 2200kbps VBR, but some titles whose source quality merits it have also been encoded at 3400kbps. The highest bitrate encodes are fit into 720x480 non-square pixels (the usual 1.2 PAR for widescreen content, 0.9 PAR for 4:3), but optimum encoding at lower bitrates is achieved with fewer pixels. Encoded films are normally at 24fps to match the source, while shot-to-video and mixed material is de-interlaced to 30fps (or 25fps for PAL content).

Netflix has been using anamorphic video all along, which I think is an underused feature of Windows Media and other formats. When you’re limited to 720x480 pixels, you want to encode all the pixels you’ve got, without having to synthesize any extra.

Second Generation Encoding

The new Silverlight player (that some users are helping us test as I write) uses VC1 Advanced Profile encoding with PlayReady DRM. A key property is that each GOP header includes frame size and resolution, which allows us to assemble a stream on the fly from different bitrate encodes as your broadband bandwidth fluctuates. (Another key feature is more coverage, including Intel Macs and Firefox users.) We expect to switch completely to the new player later this year.
The VC1 encoders are more efficient than the WMV3 encoders, so we are currently encoding VC1AP at slightly lower birates: 375, 500, 1000, and 1500kbps, all square pixel. At some point we are likely to add a couple more resolutions of non-square pixel encodes capturing the original pixel-aspect-ratio of the source.
We are also re-wrapping the VC1AP encodes in WMDRM10 for CE devices, which will gradually switch to the more efficient encodes in future firmware upgrades.

This is a great example of the improved efficiency of the VC-1 Encoder SDK and tools based on it. Not all Windows Media/VC-1 encoding is equal; the latest tools can offer a very meaningful reduction in bitrate required for a give quality level, improving user experiences and the cost of delivering the content.

The new encodes are backwards compatible to older hardware and software encoders, so even older devices can take advantage of the improvements.

High Definition Encodes

Today we have rights to deliver about 400 streams in HD (720p). More titles will be added over time. We experimented with first-generation WMV3 encodes at 4000kbps and 5500kbps, but settled on second-generation HD encodes with VC1AP at 2600kbps and 3800kbps, which extends their accessibility down to lower home broadband connections. As with SD, encodes of film material are at 24fps, and encodes of shot-to-video material are at 30fps (or 25fps for PAL), rather than the 60fps that would come from a Blu-ray disc - we judged the 60fps content as too expensive of bandwidth for now. In general, these encodes are definitively better than SD, but won't challenge well-executed Blu-ray encodes - that would require a bitrate out of reach for most domestic broadband today. We believe Moore's law will drive home broadband higher and higher enabling full 1080p60 encodes in a few years.

There’s codecs for you – a good 720p experience at 3800 Kbps, which is probably a little below the average 480p bitrate used for MPEG-2 on the DVDs that fly around the nation in those Netflix envelopes.

I’m not that worried about 1080p60 encoding myself. The vast majority of fictional content is shot 24p, including 99% at least of Blu-ray discs. While Blu-ray players may output 1080p60, the encodes, and hence bitrate requirements, are still 1080p24. I find that typical film content in VC-1 wants around 6-8 Mbps for a 1080p24 experience enough better than 720p to be worth the trouble.

I just got my Xbox set up to handle the Netflix streams – now I’ve got to go check out some of those 400 HD titles! Hmm. Pan’s Labyrinth and Heroes.

Anyway, it’s great stuff. I love it when partners can share this kind of detail about what they’re doing.

You have to be a Netflix subscriber of course, but the content is commercial-free and much higher quality than the various free video services. They encode up to a full 720x480, in proper 4:3 or 16:9, at a range of bitrates. The higher rates definitely outperform standard def cable and satellite in video quality.

Lots of good stuff in the press release, but I had a couple of things I wanted to call out (bolding mine).

Netflix, Inc. (NASDAQ: NFLX), the world's largest online movie rental service, today announced it has begun the deployment of Microsoft Silverlight to enhance the instant watching component of the Netflix service and to allow subscribers for the first time to watch movies and TV episodes instantly on their Intel-based Apple Macintosh computers. The deployment, which will initially touch a small percentage of new Netflix subscribers, is the first step in an anticipated roll-out of the new platform to all Netflix subscribers by the end of the year.

Note that this won’t be immediately available to all customers.

Silverlight is designed for delivery of cross-platform, cross-browser media experiences inside a Web browser. It is expected that Netflix members who watch movies and TV episodes instantly on their computers will enjoy a faster, easier connection and a more robust viewing experience with Silverlight, due to the quality built directly into the player. Among the viewing enhancements with the new player is a breakthrough in timeline navigation that vastly improves the use of fast-forwarding and rewinding. The new Netflix player takes advantage of PlayReady DRM, which is built into Silverlight, for the playback of protected content on both Windows-based PCs and on Macs. That had not been possible with previous generation technologies.

This is one of the first big rollouts of PlayReady with Silverlight in the USA. When you see the new navigation, it’s a real testament to the difference between a media player plugin like the WMP OCX, and a rich application runtime that plays media well like Silverlight.

The rich user experience support and DRM, what else does Silverlight bring to the equation? One big thing is that PlayReady DRM supports http delivery of pre-encrypted content, unlike other platforms that require a proprietary streaming server and protocol be used to deliver protected content, applying DRM on the fly. Pre-encrypted content is both more scalable (since the encryption processing is only done once, not per user), and more secure (since an in-the-clear media file doesn’t need to be transmitted and stored on the server).

There’s been two big technical challenges in web video for the last dozen years or so.

1. How do we deliver the best experience to each user, in a world of variable bandwidth and hardware power
2. How do we deliver that experience as cost effectively as possible, enabling new business models in video distribution.

We’re at the point now where a user with a fast connection and a decent computer should have a darn good video and audio experience. Netflix is a testament to what’s possible as the cost to deliver a MB of video gets low enough, from a business perspective. While finally delivering a user experience better than VideoCD.

• There's a new scaling algorithm that's a lot faster and much higher quality - a nice compromise when you can get it!
• Additional optimizations have been done for the VC-1 video decoder, so playback will be faster. The gains are biggest for content using B-frames.

Since the default Expression Encoder templates have "always on" scaling and its presets use B-frames by default, the above will provide a nice performance boost for existing content. The above are "always on" features - you don't need to update anything to take advantage of them. So expect smoother frame rates on lower-end machines.

Further information:

• Silverlight 2 breaking changes .doc file
• Silverlight 2 ReadMe
• ScottGu's Blog post on RC0

EDIT: Added ScottGu link

• A new Silverlight 2 base player using .NET. Among other things, this will enable players that display the video at 100% scale by default, improving quality and performance.
• New A/B compare modes. Awesome stuff for high-touch encoding and codec tweaking
• Some even more VC-1 advanced options for High Codec Nerditry.
• The first public release of Microsoft H.264 compression technology.

Anyway, a lot of good stuff for a SP1. I'm looking forward to it.

The big news is that, yes, we're going to add support for MPEG-4 to Silverlight, in the version coming after the fast-approaching fall release of Silverlight 2. Specifically, this will be H.264 for video and AAC for audio.

Why?

So, why MPEG-4 support in Silverlight? It's pretty straightforward, really. We have customers with libraries of H.264 content they wanted to publish to Silverlight, but didn't want to reencode to VC-1. Silverlight's strengths go far beyond media playback, and customers wanted the choice to deploy a wide variety of existing content within Silverlight. Silverlight aspires to provide as much choice as feasible as to how Silverlight can be authored and delivered. H.264 support is something we'd considered for past versions, but there were higher priority features we needed to deliver first. Silverlight 2 provides us a very rich base for delivering web apps, so we can start spreading our wings a bit.

What?

We've got more technical details we'll be sharing at IBC and later, but today, I'll just quote the details from Scott Guthrie's interview:

PressPass: Will you be showing or announcing anything new at the IBC conference this week?

Guthrie: Yes. At IBC 2008 we will be demonstrating a technology preview of H.264 video and Advanced Audio Coding (AAC) audio playback support in Silverlight, and H.264 authoring using Microsoft Expression Encoder and Windows Server 2008 for delivery. Until now, Silverlight has supported the SMPTE VC-1 and Windows Media formats, as well as MP3 for audio, enabling customers to take advantage of broad support across the Windows Media ecosystem, including third-party tools, service providers and content delivery networks.

We’ve always wanted Silverlight to support a variety of formats, so today we’re announcing that H.264 and AAC support will be available in a future version of Silverlight, which will offer content owners greater flexibility and choice to deliver video and audio.

PressPass: Historically, people have associated Microsoft with VC-1. Does this signal a change in direction?

Guthrie: No. Although we have been working with VC-1 for some time, it’s not widely recognized that Microsoft has also been an active participant in the standardization of H.264/MPEG AVC for many years, and we’ve included H.264 support in several Microsoft products. Microsoft’s Gary Sullivan was the chairman of the Joint Video Team (JVT), which developed the H.264 standard, and he recently accepted an Emmy Award on behalf of the JVT.

PressPass: Does this mean that Silverlight is moving away from Windows Media?

Guthrie: Not at all. This is about offering our customers more choice. Media producers and distributors around the world have enjoyed the high quality, flexibility and affordability of Windows Media formats for over a decade. As a testament to its pervasiveness, Windows Media can be found in almost every conceivable media scenario from desktop home video to feature films and TV broadcasts.

Again, don't read this as as a big change around our strategy for media formats. We've long-supported WMV and MPEG-4 side-by-side in products like Xbox and Zune. As I posted last week, Microsoft has been contributing to H.264 since its inception.

This doesn't represent any change in our support for Windows Media. Windows Media continues to work well for today's Silverlight customers. I expect (not a goal, just a prediction) that the majority of Silverlight content will remain in WMV well after we release MPEG-4 support.

In particular, VC-1 will likely remain the codec of choice for HD for some time. Comparing VC-1 Advanced Profile to H.264 High Profile with all the bells and whistles turned on, VC-1 only needs about half as many MIPS per pixel for playback. This won't matter as much for lower resolution content, or podcasting stuff that's in the simpler Baseline profile, but makes for a big reduction in system requirements for 720p and higher.

Where/When can I learn more?

If you're at IBC, swing by! If you want a head start on using MPEG-4 in Silverlight, feel free contact me directly. And we'll have plenty of more technical info to share down the road a bit.

So, I was really excited that our own Dr. Gary Sullivan, chairman of the Joint Video Team, was on hand last week to receive a 2008 Primetime Emmy^® Engineering Award for developing H.264/MPEG-4 AVC High Profile. In the words of the Academy:

Joint Video Team Standards Committee (JVT) for the development of the High Profile for H.264 / MPEG-4 AVC. The High Profile enables high definition images in the H.264 video coding system used today to deliver HD video over satellite and cable TV as well as Blu-ray Disc. The JVT is a standardization team comprised of members from the International Standardization Organization (ISO), the International Electrotechnical Commission (IEC), and the International Telecommunication Union (ITU).

The award recognized Broadcom, NTT DoCoMo, Dolby Laboratories, Fraunhofer Heinrich Hertz Institute, Microsoft, Motorola, Panasonic, Sony, and Thomson as contributing companies, as well as standards organizations ISO, IEC, and ITU.

Gary has long represented Microsoft as as chair and co-chair of the Joint Video Team that coordinated the H.264/MPEG-4 AVC effort between ITU-T VCEG and ISO/IEC MPEG, amongst his many other efforts like DXVA; his page on Wikipedia has more details.

I suppose some people might be surprised at Microsoft's inclusion here, but they shouldn't be. While Microsoft has a long heritage of making our own ground-up codecs like VC-1, we've also long been involved in standards-based codecs, including the original MPEG-4 Part 2 as well as MPEG-4 Part 10/H.264. And we've got a number of products that incorporate H.264 today, including Mediaroom, the Zune 4/8 and 80, and Xbox 360. H.264 are and VC-1 are both great codecs, with somewhat different sweet spots, and we use either or both as appropriate in our products, platforms, and services.

The picture above has, left to right: Malcolm Johnson (director of the ITU standardization bureau), Ajay Luthra (JVT vice-chair from Motorola), Gary Sullivan (JVT chair from Microsoft), Thomas Wiegand (JVT vice-chair from Fraunhofer HHI), and Scott Jameson (chair of ISO/IEC JTC 1)

And a few other articles/posts about the award:

ANSI

EE Times

Dolby

Minoru "Mick" Etoh's blog (who represented DoCoMo at the ceremony)

He's started off with a series of Olympics posts.

NBC Olympics 24/7 - a great roundup of information and details

NBC Olympics video without Silverlight? - showing how to play back the Olympics without Silverlight on Windows (and why you'd want to use the Silverlight version if given a choice).

Why no full screen mode in the NBC Olympics player?, citing this silverlight.net forum discussion including information from our own Tom Taylor.

Hopefully he'll inspire me to get back at it...

What I haven't talked a whole lot about in too long is our support for authoring Silverlight video experiences on the Mac. While our own Expression Encoder 2 is Windows-only, we work with partners to provide WMV and VC-1 encoding on other platform. Telestream has been a great partner here, with their Flip4Mac QuickTime component and their Episode stand-alone compression tool (originally known as Compression Master, and which came along with their acquisition of Popwire).

Flip4Mac is a testament to both the portability of the Windows Media technologies and the flexibility of Apple's QuickTime architecture. It works as a QuickTime component, enabling QuickTime to play Windows Media content like any other supported format, with full access to all of QuickTime's features like hardware accelerated full-screen playback (all supported in the free version). For example, in the QuickTime Pro Properties window:

Looks just like any other QuickTime file. Even the ASF metadata shows up.

Flip4Mac also supports content authoring in the paid version, as well as import into content creation apps like Final Cut Pro. It's got pretty deep control, even exposing features like B-Frames that'd take a registry key with Windows Media Encoder.

So, how about a quick walkthrough through the Flip4Mac encoding settings?

Flip4Mac is accessed like any other exportable format from QuickTime (like MPEG-4). It also comes with a bunch of presets for typical scenarios.

It has the normal set of basic video export features you'd expect, nicely Macified. I appreciate the "Size: Current" so I don't have to always type in the frame size of the source when I'm not scaling.

The Advanced button reveals some deeper options, including a complexity control for speed/quality tradeoffs and B-Frame distance. You can also set Flip4Mac to deinterlace interlaced sources, or pass interlacing on through to the final encode. Since it can both encode and decode interlaced VC-1, Flip4Mac can be a great way to transfer editable 480i video sources over slow connections. It can match DV quality in 20% the data rate.

For Silverlight audio, Flip4Mac supports both WMA 9 Standard and 9 Professional. It doesn't encode the 32-96 Kbps WMA 10 Pro low bitrate modes that Silverlight 2 will be able to decode, but does have the full variety of WMA encoding modes.

And the coolest new feature of Flip4Mac: built-in Silverlight templates! This lets you build a complete Silverlight player (using Expression Encoder templates we provided) directly from any app that support QuickTime export.

EDIT: A correction was made; turning off Fast Cache in Windows Media Services does not have any effect on webcasting latency.

So, Streaming Media is doing a special "Europe edition" of Streaming Media, and I'm doing an article about webcasting for it. I've getting a bunch of questions about how to deliver low-latency live streaming to Silverlight, and so with their permission, I'm excerpting this section on that topic.

While by default, Windows Media can offer 15-20 second end-to-end delay, it's possible to drive it down to 2-3 seconds with best practices on a good network, and we're looking at what we can do to push it to below even that. The critical thing is to tune the encoder, server, and player latency together.

There's an older but more detailed "Reducing Broadcast Delay" document on this topic over at the Windows Media portal that may be worth perusing.

Low Latency Webcasting

Latency is the measure of how much time goes between when video enters the encoder and leave the user display. Latency is something that doesn’t matter at all in some markets, and matters a lot in others.

The reason we have latency is buffering, and the reason we have buffering is for quality and reliability. By having the server wait several seconds after a video stream is received before sending it out, it’s able to support more peaks and valleys in the data rate, and makes it possible for a dropped packet to be detected and resent before its needed. In the same way, buffering in the player lets it average out data rates and recover dropped packets as well.

Large buffers were extremely important in the modem era, and are still useful in many kinds of networks today. The defaults are good for delivering high quality content over a variety of networks. But when minimizing the latency is important, and reliable networks are available, the end to end delay can be reduced substantially.

Since total end-to-end latency is the sum of the encoder latency, server latency, and player latency, plus how long it takes the packets to travel between each of those, improving latency requires tweaks to the encoder, server, and player in parallel.

 

Encoder

On the encoder side, reducing buffer size reduces startup latency. WME and Expression Encoder both enable you to go down to one second.. Using Lookahead or Lookahead Rate Control will increase latency beyond the buffer value (typically about another half a second for LRC).

For audio, Windows Media includes the “WMA Low Delay” audio codecs from 192-64 Kbps which provide lower latency than the normal WMA modes. If you’re targeting Silverlight 1.0 or WMP 10 or earlier, you’ll want to use that for low latency. If you can require WMP 11 or Silverlight 2, you can use the lower delay yet WMA 10 Pro codec at 32-96 Kbps.

 

Server

WMS features like Advanced Fast Start can dramatically reduce latency for on-demand content, but don’t apply to live webcasting as the server has to wait for content to arrive from the encoder it in real time, and so can only play out at real time.

Server buffering can be turned off entirely in WMS for Windows Server 2003 and 2008, yielding a significant drop in latency.

EDIT:

The previous version of this post suggested that turning off Fast Cache would help webcasting latency. On further research, this turns out not to be the case. Disabling Fast Cache will slow down on-demand startup time, but will have no effect one way or the other for live content.

Player

Normally the player isn’t under control of the streamer. By default WMP dynamically picks an optimum buffer size based on its measurements of network and stream performance. However, it’s possible to lower the buffer size in the player’s options. This can help reduce latency when watching streams with a good connection, but could produce pauses in the video when watching video from the general Internet.

Silverlight makes the player buffer size a controllable parameter, so an optimal setting can be applied for the content, and even adjusted on the fly. This is controlled by the BufferingTime parameter in a Silverlight MediaElement.

Read the article yourself, but here's a few choice quotes from it:

• "NBCU plans to offer 3,600 hours of live programming from Beijing. That’s 212 live hours for each of the 17 days of the Olympics... In addition to the sheer volume of live content to be delivered—three times what was offered in 2004— what’s notable is that most of NBCU’s live programming—2,200 hours—will be delivered online at NBCOlympics.com."
• "The 2008 Summer Games in Beijing will mark the arrival of streaming as a viable alternative to the Olympics’ television broadcast. This summer, NBCOlympics.com will offer 4,400 hours of on-demand streaming in addition to its 2,200 hours of live programming, making the Beijing Olympics the most ambitious streaming media project in history."
• "To help meet that challenge, the NBCOlympics.com player offers a “metadata overlay” feature, which allows the player to display transparent data and navigation tools over the video window. This enables users to access statistics and other data without covering up, pausing, or leaving the primary video display. For example, play-by-play announcers’ dialogue can be keyed into an XML data stream, then rendered as a timecoded, scrolling text caption that transparently overlays the bottom of the video display. The player also enables the TiVo-like experience of pausing, rewinding, and replaying content, and these two features together allow viewers to use either the timecode or the play-by-play captioning to rewind to a specific point in the on-screen action and replay it."
• "A slew of DRC-Stream software and encoder boards from Canada-based Digital Rapids are being deployed in Beijing to populate NBCOlympics.com’s encoding farm, but other than committing to VC-1, NBCOlympics.com has yet to confirm encoding bitrates, frame rates, or frame sizes. (Without offering more specifics, Miller says NBCOlympics.com will be streaming through a managed bitrate solution to optimize the user’s connection, with a target maximum bitrate of 650KB/sec.) Digital Rapids is also supplying software to enable transcoding from other digital media formats into VC-1.Miller promises hundreds of hours of online HD video..."

Note that there's already a bunch of content up at NBCOlympics.com if you want to get an early taste of what's in store on 8/8/08.

Here it is

Sample projects

It includes a number of sample projects (quoting from the help file). These are a good jumping off point for various scenarios.

Simple encode

The simple encode sample is a console application that demonstrates how to encode a file from a console application when showing progress. To use it, just run the built application (Simple.exe) at a command prompt.

Asynchronous encoding

The WPFEncode application demonstrates how to encode a file from a UI application when displaying the progress of the encoding process. To use the application, click the Browse button to point to a file that you want to encode and then click the Encode button to encode it. When the file is encoding, a bar displays the encoding progress.

When you press the Encode button, the application creates a thread to perform the encode to make sure that the UI thread is not blocked. The progress events are called from a non-UI thread. Consequently, we use the Windows Presentation Foundation Dispatcher to marshal the updates to the UI thread.

If you try to close the application when the encoding is still occurring, the application prompts you to confirm that you want to stop the process. If you confirm, the application stops the encoding thread but delays closing the application until the thread actually stops.

MediaInfo

Publishing plug-in

Publishing plug-ins enables you to take video and Microsoft Silverlight template assets that you have created as part of an encoding job and then do something with encoded assets. For example, you can upload the media asset by using FTP, WEBDAV, or METAWEBLOG API, publishing the asset to an asset management system or web service, or creating a data CD.

The following code sample demonstrates how to use a publishing plug-in to add the output of an encode job to a Zip file by using the Open Packaging Convention components found in Microsoft .NET 3.0.

A publishing plug-in is required to derive from the PublishPlugin class, which in turn derives from the EncoderPlugin class:

Template plug-in

The template plug-in lets you customize template settings and interact with the job through the IPluginHost interface. The plug-in also lets you add or remove files from the template output when the template is published.

The following code sample demonstrates how to add the default template parameter control to Microsoft Expression Encoder with only several lines of XAML. It also implements a custom script command creation feature that shows how the template plug-in can interact with the current job by adding a script command at the current play head position of the selected MediaItem.

Within Expression Encoder, every template receives a template plug-in. If the template doesn’t specify one, a default one will be assigned. A template plug-in derives from the TemplatePlugin class, which in turn derives from the EncoderPlugin class.