New Vista GUI Stuff For Devs

The Discussion

• 

  The problem I see with this compositing technology is it creates a large problem in order to solve a very small problem.  You now need entirely new display drivers throwing out the old ones that have had years to stabalize, application compatibility takes a hit especially screen readers, and you need a complex and potentially unstable system to actually do the compositing.  All of these problems just to fix the issue of rare drawing artifacts and add a bit of eye candy.  It doesn't seem worth it.
  
  Hopefully Microsoft has bigger plans for DWM 2.0 and something more intuitive than the infamous Flip 3D.

• 

  DigitalDud wrote:

  You now need entirely new display drivers throwing out the old ones that have had years to stabalize

  
  
  The video card industry produces new cards so quickly that I don't think you can say that their drivers need time to stabilize. In 6 months there will be a whole new wave of cards, and the current "stable" drivers will be useless for them.
  
  On the other hand, most video card manufacturers now have a unified driver architecture. It's not uncommon for the release notes of new drivers to say something like "x% increase in performance for games a, b, and c on the following cards...". So, in theory, the older cards will benefit from added stability as time goes by because there is continuous active development and refinement on that codebase as the devs squeeze out every bit of speed and efficiency they can out of the cards.

• 

  The DWM makes window dragging look great, and the Glass and Flip3D are neat looking, but I'm really dissapointed with the quality of window redrawing while resizing a window, especially for windows with client area glass like Windows Media Player. Resizing WMP on my system leaves behind an ugly black ghost of the glass area at the bottom of the window trailing behind as it attempts to keep up with the window redraw. Frankly it looks much better with the old non-composited redraw behavior..
  Is this going to get fixed anytime soon or in the next version of Windows?
  

• 

  DigitalDud wrote:

  The problem I see with this compositing technology is it creates a large problem in order to solve a very small problem.  You now need entirely new display drivers throwing out the old ones that have had years to stabilize, application compatibility takes a hit especially screen readers, and you need a complex and potentially unstable system to actually do the compositing.  All of these problems just to fix the issue of rare drawing artifacts and add a bit of eye candy.  It doesn't seem worth it.
  
  Hopefully Microsoft has bigger plans for DWM 2.0 and something more intuitive than the infamous Flip 3D.

  
  
  It doesn't seem like WDM should be much more complex than the old method.  In fact it should provide a more stable and robust system.  I wasn't sure what you meant by screen readers but I'm guessing you mean screen capturing software.  The current screen capturing software (especially video) won't work fantastically b/c it wasn't designed for WDM (athough... they never worked that well in the first place).  However capturing screens of the desktop shouldn't be too hard in Vista since it's all rendered in 3D.
  
  Using hardware accelerated graphics opens up many possibilities that are otherwise very slow or impossible to implement.  Occlusion, transparency and animations are all easily done very quickly through hardware.  The point is that, many computers now have decent graphics cards that are severely under utilized.  With WDM and WPF the entire computer is utilized providing performance increases, and of course it provides a lot of graphical freedom for applications.
  
  Compatibility is always an issue transitioning between technologies but it shouldn't stop new innovations from being developed.  Plus these transition problems are very short lived.

• 

  Jhaks wrote:

  
  
   I wasn't sure what you meant by screen readers but I'm guessing you mean screen capturing software.  The current screen capturing

  
  
  He means something like 'Narrator' on XP (Accessories->Accessibility).  Try using windows with your eyes closed.
  
  

• 

  I don't know if this interview was done before or after CES but when Kam and Rory where talking about how if you're designing an control panel extension you might want to make it look exactly like the rest of the OS (something I completely agree with) but otherwise most people want to design something that looks slightly different as a form of branding or maybe they just think they can a better job themselves.
  
  Of course you can go overboard with this but the newest Yahoo Messenger is the best example of using the new tools to make something completely unique but still functional and not too flashy.
  
  If you haven't already, watch Jeff and Karsten's video over at on10.net
  http://on10.net/Blogs/larry/yahoo-messenger-on-wpf/

• 

  DCMonkey wrote:

  The DWM makes window dragging look great, and the Glass and Flip3D are neat looking, but I'm really dissapointed with the quality of window redrawing while resizing a window, especially for windows with client area glass like Windows Media Player. Resizing WMP on my system leaves behind an ugly black ghost of the glass area at the bottom of the window trailing behind as it attempts to keep up with the window redraw. Frankly it looks much better with the old non-composited redraw behavior..
  Is this going to get fixed anytime soon or in the next version of Windows?
  

  
  Completely agree.  I was dissapointed that the contents weren't double-buffered as they are in OSX to avoid any kind of flickering, but it's really surprising how bad it looks - WMP11 is the biggest offender. 
  
  I would hope that this is a driver issue, but MS hasn't exactly paid that much attention to the niggling details like this in the past.

• 

  Nitz Walsh wrote:

  

  DCMonkey wrote: The DWM makes window dragging look great, and the Glass and Flip3D are neat looking, but I'm really dissapointed with the quality of window redrawing while resizing a window, especially for windows with client area glass like Windows Media Player. Resizing WMP on my system leaves behind an ugly black ghost of the glass area at the bottom of the window trailing behind as it attempts to keep up with the window redraw. Frankly it looks much better with the old non-composited redraw behavior.. Is this going to get fixed anytime soon or in the next version of Windows?

  
  Completely agree.  I was dissapointed that the contents weren't double-buffered as they are in OSX to avoid any kind of flickering, but it's really surprising how bad it looks - WMP11 is the biggest offender. 
  
  I would hope that this is a driver issue, but MS hasn't exactly paid that much attention to the niggling details like this in the past.

  
  
  Well, this isn't really the fault of the DWM system in Vista. Admittedly, its marginally better when running in classic mode (no black section, but still slow to resize), but the root cause seems to be the fact that if you resize WMP quickly on a fairly slow machine, it becomes apparent that the black section you're talking about is in fact the 'classic' black non-glass WMP controls, buttons and all, still being rendered in the background!
  
  This is more a case of poor software development than anything else.
  
  If you try resizing with a proper WPF app, such as the New York Times reader, you'll find it a much better experience, because remember that although you are running a composited desktop, inside the composited window, unless the application is WPF, it's all still GDI based rendering.
  
  I dont think Vista ships with any WPF applications off the top of my head. In fact, for that matter, i don't think it ships with any .NET 2.0 executables either.
  
  I think we all often forget that what we currently know as 'Vista' isn't the full picture by any means. WPF is a key part of it, an amazingly powerful platform, and is currently 99% unused, even in Vista itself.  A few months down the line, we'll have DX10 graphics cards, designed with WPF in mind, accellerating even more of the WPF framework than we do now with DX9. It's going to be big, and what we've seen so far is nothing.

• 

  Hmmm... I'll stop blaming my graphics card drivers, then!   Here I thought maybe I was the only one.  But I do have a pretty fast machine overall, and even still, I agree--WMP looks quite bad when resizing.  I've been telling people that Aero would get rid of window-tearing... guess not!  Maybe it is a problem with WMP itself, but I don't think anyone cares about whose "fault" it is--they just want to see the graphics drawn properly.  The resizing in WMP is definitely not a good experience.
  
  As for WPF apps in Vista, I thought Photo Gallery is one, but I'm not sure.  I agree about Yahoo's new Vista-specific Messenger, though--that looks slick!  And not just in an eye-candy sense--it really does add to the experience.  It's a perfect WPF showcase app... and it's rather sad that it didn't come from Microsoft.  In fact, I think Nitz Walsh and I had this discussion about a year ago on Channel9--that Microsoft really needs to be the one driving the graphics ability present in the system, partly to set the standard for others, partly to show faith in their own new technology, and partly to show what the system can do.  Not to slag against Microsoft, but it is a bit disappointing to watch, in this example, Yahoo do what the Windows Live Messenger team should have done first.

• 

  I guess it depends on what you understand 'tear-free' to be. I think the definition meant here was 'in sync with your displays v-sync and wthout a slow redraw of the background window', which the DWM does pretty well. As you say though, resizing is not a good experience, but I think any OS is much the same.  In fact, IIRC, Mac OS X was particularly bad at this, despite having  beautifully smooth window dragging even on a GeForce 2.
  Moving a window, with the DWM, just means moving a textured surface over the one beneath it. In GDI, the surface was moved, but the one below has the extra step of having to be redrawn again.  Resizing a window is a different problem, becuase it means all the contents inside have to reflow and redraw.  This involves quite a computational and rendering overhead, especially in rich evolved GDI apps like Windows Media Player.
  
  Photo gallery appears to be a standard win32 app according to its files, although it does make use of 'glass'. I certainly agree about Yahoo Messenger, quite why Microsoft didn't have a few impressive WPF apps lined up is strange, but haven't we seen the same pattern with .NET 2.0?, at least in terms of mass-market apps.
  
  I think Microsoft is positioning itself to provide the best development platforms for these future applications, and I also guess there must be just too much work invested in the older win32 architectures and applications (think: Office). In terms of development tools, I dont think anyone can choose a better environment than VS, and Microsoft seem to be heading for it again with Expression Blend,  the WPF development tool, which *is* written  in .NET and uses WPF, and is really quite amazing, as development tools go, already.
  
  Having said that, it seems like a big oversight that there were no WPF apps shipping with Vista, and it could have made a lot of the initial reviews a lot more favourable for its Aero UI, especially when inevitably compared to Mac OS X's UI (to which i think everyone will agree, WPF is technically superior)
  Having said that, they may have a few ready and waiting for the January 30th launch
  

• 

  MarkPerris wrote:

  Having said that, it seems like a big oversight that there were no WPF apps shipping with Vista, and it could have made a lot of the initial reviews a lot more favourable for its Aero UI, especially when inevitably compared to Mac OS X's UI (to which i think everyone will agree, WPF is technically superior)
  Having said that, they may have a few ready and waiting for the January 30th launch
  

  
  Agreed.  The Yahoo! presentation was quite impressive though, it's given me a lot of hope that future WPF on Vista will really start to impress.
  
  But as well, I can't understand why something like the Photo management app in Vista isn't WPF.  Or how about a WPF-only Media Player 11 version?  That would be basically a no-brainer it seems.  I think people that play with Vista at a store will likely feel that the experience is grossly overhyped, simply because none of the apps really take advantage of the underlying technology.  MS has to be the one to set the bar here.

• 

  Awesome!
  Where can we download the sample code shown in the demo?

• 

  Speaking of screen captures. Why dont the interviewer switch to a video screen capturing tool eg: http://www.microsoft.com/windows/windowsmedia/howto/articles/screencap.aspx when doing the PC/laptop demos as often the focus is off or screen is blurred when for source code and app are being displayed.
  Together with a little bit of editing of both the handy-cam footage, it would create a much better viewing experience for the audience.

• 

  Can Rory not Sharon-Stone so much in the next video Thanks.
  

• 

  Minh wrote:

  Can Rory not Sharon-Stone so much in the next video Thanks.
  

  
  
  NFK.  I was cringing too much to watch the video. 
  

• 

  DCMonkey wrote:

  The DWM makes window dragging look great, and the Glass and Flip3D are neat looking, but I'm really dissapointed with the quality of window redrawing while resizing a window, especially for windows with client area glass like Windows Media Player. Resizing WMP on my system leaves behind an ugly black ghost of the glass area at the bottom of the window trailing behind as it attempts to keep up with the window redraw. Frankly it looks much better with the old non-composited redraw behavior..
  Is this going to get fixed anytime soon or in the next version of Windows?
  

  
  
  The redraw flicker and artifacting was the first thing I noticed when I installed Vista RC1.  I am surprised this issue was not dealt with.   I guess it sort of makes sense because IE is a win32 control, not a native WPF.  It really makes me not want to use an IE control in my WPF apps.   The IE team better get a WPF version out there because the next version of Firefox has HW accelerated vector render engine.

• 

  Sorry for coming to this thread so late after the interview. Rory promised me a "heads up" before it got posted, but I didn't end up seeing it until now.
  
  I know some folks are asking for the source code for the demo that was shown, you can download it from here. This sample was based on some code that was written for other demos we did internally, so if the part of code to extend glass into the window looks familiar, that is why.

  There's been some lively discussion on this thread, so I'll take a moment to address some of the comments.

  DigitalDud mentioned the need for new display drivers to do composition. I've talked about the benefits of composition in the past, but it's worth discussing the new driver model briefly. The new driver model is really a separate thing from Desktop Composition.

  With the new driver model, the biggest change was moving a great deal of code out of kernel mode into user mode. There were meny motivations for this - including, but no limited to One motivation behind this was that video drivers are increasingly doing a great deal more language processing than they had in the past, due to new functionality like shader processing at the pixel level and for geometry.  Moving this processing up into user mode means it's less likey that an application can cause the driver to crash in kernel mode - in other words, far fewer blue screens.

  Desktop composition does take advantage of the new driver model, in that it enables better support for more than one application using the GPU simultaneously, sharing graphics memory, and managing graphics memory together.

  DCMonkey asked about resize behavior. The biggest challenge with improving resize behavior is application compatibilty. For new apps, like those written in WPF, we had more latitude to change what happened in the application by default when you resize it. In WPF, you don't manage your DC or your DX surface directly - the framework does that for you. So we can make sure that the framework does something "DWM friendly". For existing Win32 applications, and even some Windows Forms applications, we need to make sure that we don't do things that change the expectations that applications already have. Some applications do processing on the resize loop by listening for and reacting to WM_SIZE messages. Sometimes those applications don't get that work done in less than 16ms (which would be required to do a resize at 60FPS assuming no system overhead). And of course, the system does have overhead so the applications don't really get their whole 16ms - what they get varies from system to system based on hardware.

  There are a number of approaches that can be taken to keep these things in sync and looking smooth - one way to do it is to not update the window frame until the contents of the window have drawn. The downside to this, is that then the window updates will lag behind the mouse-pointer movement. Apple's OSX exhibits this behavior pretty commonly (at least Tiger does on the Mac Mini we have in the office). Another approach is to have the window frame track the mouse, and use a default brush to fill the new window area until the window gets around to repainting - that's what we do in Windows Vista.

  If the application chooses to draw glitchy stuff in that resize loop, there's not a lot we can do. The one thing we can do is focus on making sure that the amount of processing that happens in DWM.exe during resize is minimal, and we spent a great deal of effort on that front.

  There was some discussion of .NET 2.0 executable shipping in Windows Vista. It's true, there are none (to my knowledge). That doesn't mean that Microsoft isn't investing in great experiences built on the .NET framework. The Max project was probably our first major public endavour in building a great user experience on .NET. That project has closed now, but there are other things happening as well. There are the Expression tools, and there's also the new Windows Live for TV Beta, which is an extension to Media Center on Windows Vista that was built using the framework. I'm sure there are more things like this coming as well.

  Thanks for all the great questions & comments on the interview. Its always fun to do these, and be able to participate in discussions like this! If y'all have more questions, I've cross-posted this note to my blog and I'm happy to take more questions there as well.

• 

  MarkPerry wrote:

  [quote]In fact, IIRC, Mac OS X was particularly bad at this, despite having  beautifully smooth window dragging even on a GeForce 2.

  
  True. Maybe it's been worked out now, but on my old Mac Mini G4 this is the one thing that sticks out like a sore thumb. It can handle window dragging and drop shadows beautifully on the Mini's miniscule 32 MB video card, but as soon as you grab a window resize handle and yank it, it's as if you just asked OS X to calculate the meaning of life. You can resize, go out to lunch and when you come back it's still figuring out how to redraw.
  
  I haven't tried Vista yet (tomorrow is the virgin tour) and I'm quite shocked about the stuff I'm reading here. Considering what Core Animation in OS X can do on a 32 MB card, and the fact that Aero asks for 128 MB minimum (or is it 256 MB?) I was expecting no less than freaking Industrial Light & Magic quality from Aero, DWM should be running circles around Core Animation -- and if what you're saying about WMP11 is true, then whoever greenlighted it for release shouldn't merely be fired, but buried. WMP11 was a downgrade from WMP10 anyway, bring on WMP12 *now*.