WPF Imaging (Channel 9)

Recently, Tim Sneath, a Channel 9 correspondent and Technical Evangelist took us over to the WPF imaging team to learn about some of the great work that's been done in that area. Substantial (and innovative) improvements have been made to the way Windows manages images and renders images and the WPF People have surfaced these improvements for us, the developers. There's been great work done in the WPF imaging APIs making it much easier to powerfully maniuplate images from managed code. Check this out.

Follow the Discussion

Follow
Oops, something didn't work.

Try again?

Sign In to be begin to follow these comments

What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in. You need to be signed in to Channel 9 to use this feature.

Getting "follow" information

Stop following these comments

Start following these comments

What does this mean?
Following an item on Channel 9 allows you to watch for new content and comments that you are interested in and view them all on your notifications page.

Removing your "follow"

Setting up your "follow"

Did you know you can
sign up for email notifications?
- Subscribe to this comments

Minh WOOH! WOOH!

Mar 06, 2006 at 2:13 PM

Interesting idea about matching the pattern of bits to a file codec. I guess the 4 CC, magic number isn't used any more?
BryanF

Mar 06, 2006 at 5:21 PM

Great video. Four quick questions:

(1) Are there any analogous WPF subsystems for other types of media (audio, video)?

(2) On a related note, what about vector images, such as SVG, WMF, or Visio diagrams? (SVG in particular)

(3) What's the installation policy with image codecs? The video mentioned than unknown image formats are passed to the application as a raw blob, but is there any service that will attempt to fetch codecs from the web, like in WMP today? Are there any special security issues when installing 3rd party codecs?

(4) Not for the imaging team, but will IE7 be upgraded to this new API? It would be great to future-proof against new codecs to avoid another wait between releases as with transparent PNGs.

Much thanks.
DigitalDud

Mar 06, 2006 at 6:11 PM

You guys need to resurrect JPEG 2000. It was so much better than any image format we have today, beautiful images, great compression compared to JPEG, loss-less mode, it was awesome. Yet it never seemed to catch on, I guess because it was so complicated. Wavelets are definetly superior to cosine transforms.
Zeo Channel 9 :)

Mar 06, 2006 at 6:16 PM

Nice to see the avalon team back on C9.....the more Vista technology videos the beter . Peggy is really cool. She's seems like a perfect candidate for the WM_IN series....but alas seeing her here on C9 is just as cool.

I love the phone ringing in the video. Another person getting voicemail because of C9 filming....hehehehe.

Charles nice camera work....although during parts of the video I got dizzy with all of the back and forth and round and round with moving between all of the people being focused on.
alan_burchill

Mar 07, 2006 at 3:07 AM

What was the url for the guys blog where he post the WIC Explorer...??

Thanks

Alan
Sepcot

Mar 07, 2006 at 7:10 AM

alan_burchill wrote:

What was the url for the guys blog where he post the WIC Explorer...??

Thanks

Alan

The WIC Explorer code is up on Robert Wlodarczyk's Blog: http://blogs.msdn.com/rwlodarc/archive/2005/09/21/472338.aspx
RWlodarczyk

Mar 07, 2006 at 7:24 AM

Indeed, that's correct. I haven't updated it for the Feb. CTP yet, but will be doing so in the next few days.
PerfectPhase "This is not war, this is pest control!" - Dalek to Cyberman

Mar 07, 2006 at 9:59 AM

With all this plugable codec options, is there going to be a way to select which is the default for an image type. I can imagine several different apps installing their own codecs for types like RAW. I find it a pain with DirectShow to change the priorty of filters registered for the same type, some work much better than others
SlackmasterK This Space Intentionally Left Blank

Mar 07, 2006 at 10:16 AM

must... install.. motors on desk...
cannot... trust... weight of equipment... to small EM currents...
therefore must remember locknut.
rasx Emperor of String.Empty

Mar 07, 2006 at 12:50 PM

This Blog entry will talk about Channel9 Videos. This completes my disingenuous exposition.

Apart from Robert Scoble’s famous tension-filled laugh (which by the way is “socially acceptable”—it just sounds tense to me—but then, again, I like “free jazz” and Sponge Bob does not), there is the continual request to not look in the camera. This advice seems strange to the philistine children of the Reagan era—especially the “technical” folks at Microsoft. But my lack of formal education in media history suggests to me that during the 1980s, the rise of the “infomercial” introduced this innovation. This is a professional camera technique that manipulates the psychological powers of the viewer.

When the subjects in camera do not look at the camera, they suggest to the viewer that they are, at best, part of the conversation—at worst (which more often) the viewer is encouraged to indulge in voyeuristic conquests just short of the pornographic experience. When the talking heads do not look in the camera, the viewer can feel like they are uncovering documentation instead of being subject to a presentation. This technique seduces the viewer into thinking they are in charge of the situation (or at least genuinely respected) when (more often) they are not. The reason why I say they are not is because there are more infomercials in “popular” media than documentaries. The arithmetic does not lie. The camera often does.
LaBomba Summer

Mar 07, 2006 at 3:34 PM

Yes,

going deep... keep them coming guys.

Good stuff.

-LB
ShadowChaser It's not easy programming with paws.

Mar 07, 2006 at 5:33 PM

:O BitmapFrame?

I must be dreaming... managed support for image animations designed right into the core of the new imaging classes, and not an afterthought? WOOHOO.
PeggiG

Mar 07, 2006 at 6:24 PM

Hi Bryan. I've inserted answers to your questions inline. Thanks for asking! - Peggi

BryanF wrote:

Great video. Four quick questions:

(1) Are there any analogous WPF subsystems for other types of media (audio, video)?

   [PG] You can use video and audio in WPF, and even integrate it with animations, images, and user interaction, but we don't have an extensibility platform for it like we do for images.

(2) On a related note, what about vector images, such as SVG, WMF, or Visio diagrams? (SVG in particular)

   [PG] The only discovery mechanism we have for 3rd party formats currently is for images. This is largely driven by the need to support Raw image formats, because new ones are created fairly frequently. (Of course, it isn't limited to Raw formats, but it is currently limited to image formats.)

(3) What's the installation policy with image codecs? The video mentioned than unknown image formats are passed to the application as a raw blob, but is there any service that will attempt to fetch codecs from the web, like in WMP today? Are there any special security issues when installing 3rd party codecs?

   [PG] When you install an image codec, it gets registered in the registry and we match a pattern in the image file against the registry entry to identify the codec when an image is requested to be loaded. If the image format is unknown (no matching codec on the machine), today, we just return an error saying the format is unknown. (It's actually unknown metadata formats that we treat as a BLOB; that wouldn't really work for image formats.)

   - Having a service that attempts to retrieve a codec from the Web would be a good idea, and one that we'll consider for future versions. We do need to be careful about installing 3rd party codecs because of the security risks. It's one thing for a user to consciously decide to install a 3rd party codec, but if we (Microsoft) install it, we have to have a way to make sure it's really secure. Otherwise, we'd be putting our users at risk.

   - There are always security risks when installing a codec. We've gone to great lengths to make the codecs we ship with WIC/WPF as secure as possible, but we currently do not have a way to ensure that 3rd party codecs are secure. That's why they require admin priveleges to install them. That's not really a solution, though. In the future, we plan to have a certification program for image codecs that will require a specified number of hours of "file fuzzing" for any 3rd party image codec to ensure at least a baseline of security. In the meantime, if an image codec is discovered to have a security vulnerability, we will add it to a blacklist that we check before loading any codec. Blacklisted codecs will never be loaded. The codec manufacturer will have to release a security patch to reenable their image format if this occurs.

(4) Not for the imaging team, but will IE7 be upgraded to this new API? It would be great to future-proof against new codecs to avoid another wait between releases as with transparent PNGs.

   [PG] Not being on the IE team, I can't make any promises, but I expect that they will be using the Windows Imaging Component. I just don't know what the timeframe will be.

Much thanks.
PeggiG

Mar 07, 2006 at 6:35 PM

PerfectPhase wrote:

With all this plugable codec options, is there going to be a way to select which is the default for an image type. I can imagine several different apps installing their own codecs for types like RAW. I find it a pain with DirectShow to change the priorty of filters registered for the same type, some work much better than others

[PG] It would certainly be possible to provide a way for the user to select the default codec for an image format. In general, we don't expect there to be alot of different codecs for the same image format, but it's certainly possible.

In the developer platform itself, we don't expose any UI to the end user, but an application (or the Windows Explorer) could expose some UI to enable the user to select a default codec for an image format, and then always request that specific format for that file type. That feature isn't supported in Windows Vista but, if enough people request it, there's no reason it couldn't be supported in a future version.
ibrahim

Mar 08, 2006 at 1:31 AM
footballism Another Paradigm Shift!

Mar 08, 2006 at 3:36 AM

Does any one know where I can get the managed demo app shown in this video?

Sheva
PeggiG

Mar 08, 2006 at 11:32 AM

Right now, I just have it on my machine.
I'll see if Robert can post it on his blog.
Jsoh :::Windows 8 Style UI :::

Mar 08, 2006 at 9:14 PM

Near the end there was talk about 32bit channels. Those would be some large files. An image 8mp would be 72mb raw files. I would like to see the undo stack push and pop those!
geekling I am an artist

Mar 08, 2006 at 9:25 PM

[PG] The only discovery mechanism we have for 3rd party formats currently is for images. This is largely driven by the need to support Raw image formats, because new ones are created fairly frequently. (Of course, it isn't limited to Raw formats, but it is currently limited to image formats.)

Is this discovery mechanism exposed to developers not inside Microsoft (like me?). If so, is there any API documentation about how it works?
Hasenoehrl

Mar 09, 2006 at 12:45 AM

Is there a way to load a decoder without a pattern?
We currently use the compound document file format for our image.
We have a root CLSID in there. So is there a way to load the decoder only by file extension or a compound file root CLSID?
billh call -141

Mar 10, 2006 at 6:20 AM

Thanks for this video, Charles...these are the kind of videos that make me reconsider applying to Microsoft (yes, I admit, I've done it before, so sue me :O ). I love looking at and putzing around with file formats, and know a little about the various image formats out there.

With regards to "anti-fuzzing", my concern would extend well beyond image files, though...are you doing this for .wav formats, media formats and other types of files? I know with video streams the Media Player will usually balk in some way (with some type of "corrupted file" error dialog). It is ridiculously easy to come up with your own file formats, and as an extension of that, fiddle with the ones that are out there now.

Edit: Are there any updated graphics format pages anywhere on the internet? Here is a page of older formats if anybody is technically curious (circa 1997):

http://www.dcs.ed.ac.uk/home/mxr/gfx/2d-hi.html
RWlodarczyk

Mar 11, 2006 at 1:06 PM

The PhotoDemo is now available! http://blogs.msdn.com/rwlodarc/archive/2006/03/11/549602.aspx
PeggiG

Mar 11, 2006 at 1:41 PM

geekling wrote:

[PG] The only discovery mechanism we have for 3rd party formats currently is for images. This is largely driven by the need to support Raw image formats, because new ones are created fairly frequently. (Of course, it isn't limited to Raw formats, but it is currently limited to image formats.)

Is this discovery mechanism exposed to developers not inside Microsoft (like me?). If so, is there any API documentation about how it works?

[PG] There's no API required. We do the discovery automatically. You just have to include a pattern in the image file that uniquely identifies the image format, and register the pattern in the registry. The discovery process itself is automatic. At runtime, we match the pattern in the file with the pattern in the registry, and that's how we discover the right codec.

BTW, I said that we only use discovery for image formats. That's not quite true. We also use it for metadata handlers and pixel format converters. (But only for images.)
PeggiG

Mar 11, 2006 at 1:45 PM

Hasenoehrl wrote:

Is there a way to load a decoder without a pattern?
We currently use the compound document file format for our image.
We have a root CLSID in there. So is there a way to load the decoder only by file extension or a compound file root CLSID?

[PG] We do require a pattern that uniquely identifies the format, but the pattern can be anything you like, as long as it always apprears at the same place in the file. In fact, we strongly recommend that people use GUIDs for their identifying pattern, because that guarantees the pattern is unique. So, if the CLSID appears at the same place in each file, you can use the CLSID as the pattern.
MarClown

May 09, 2006 at 11:50 AM

Hey Channel 9 guys,

What about using a small tripod when shooting in an office ?
The topics are interesting but I'm having a bit of motion sickness here...
Your panning will be smoother and your video much smaller (you care about bandwith right ?)
duncanmak

Oct 28, 2008 at 2:59 PM

Can this video be made available again?
Martin

Nov 03, 2010 at 2:55 PM

Please re-post video. Thanks!